backup and recovery exchange 2010 dag

7/24/2019 Backup and Recovery Exchange 2010 DAG

1/12

Backup and Recovery-Exchange 2010 DAG

1

Backup and Recovery Exchange 2010 DAG

By Francis Sabarinath [frasa03]


2/12


2

Table of ContentsBACKUP AND RECOVERYEXCHANGE 2010 DAG ........................................................................................ 3

1. Exchange Backup................................................................................................................................... 3

2. Backup Sequence Diagramusing Store Writer .................................................................................. 3

3. Backup Sequence Diagramusing Replica Writer ............................................................................... 4

4. Difference between VSS backup of Exchange 2010 using Windows Server Backup and that done by

CA ARCserve Agent for Microsoft Exchange ............................................................................................. 4

5. Important Tool for Troubleshooting Exchange Backup issuesMS Event Viewer .............................. 5

6. User Problems related to Backup ......................................................................................................... 8

6.1 User Problem 1 -Exchange Database Level Backup fails ................................................................. 8

6.2 User Problem 2 -Exchange Database Level Backup fails - Client halt error .................................... 8

6.3 User Problem 3 - Transaction Logs not truncated .......................................................................... 9

6.4 User Problem 4Unable to find Exchange Database .................................................................. 10

7. Exchange Restore ................................................................................................................................ 10

8. Restore Sequence Diagramonly Store Writer is used and not Replica Writer ................................ 11

9. User Problems related to Restore ....................................................................................................... 11

9.1 User Problem 1-Restore using Recovery DatabaseHow to extract the Mailbox ...................... 11

9.2 User Problem 2Unable to find the Recovery Database ............................................................. 12


3/12


3

BACKUP AND RECOVERY EXCHANGE 2010 DAG

1. Exchange Backup

Microsoft provides 2 ways of backuplegacy streaming and VSS. Legacy streaming backup is not

supported with Exchange 2010. The VSS communicates with the Windows Server File system and with

the mass storage device driver through a third-party (or customer) provider. The backup/restore

application is also called the VSS Requestor. The requestor communicates with the VSS to obtain

information about the Exchange 2010, to command the creation of shadow copies and to gain access to

the data for backup. To support the VSS, Exchange 2010 includes an Exchange writer that it built into the

Exchange Store. The Exchange Writer coordinates with the Exchange Store, to freeze and dismount the

database before backing it up, and then to unfreeze and mount the database after the backup is

complete. Backup from the active copy uses the Store Writer, and backup from the replica (copy

database) uses the Replica Writer. It is good to understand the Backup sequence using the Store and the

Replica writer which is diagrammatically provided in the following section.

2. Backup Sequence Diagram

using Store Writer


4/12


4

3. Backup Sequence Diagram using Replica Writer

Ref:http://msdn.microsoft.com/en-us/library/exchange/aa579076(v=exchg.140).aspx

4. Difference between VSS backup of Exchange 2010 using Windows ServerBackup and that done by CA ARCserve Agent for Microsoft Exchange

The sequence of backup will be the same except that a physical consistencies check that will be initiated

by ARCserve Exchange Agent. Our agent uses the exported functions from chksgfiles.dll to check the

consistency.

The CHKSGFILES API ships as part of Exchange 2010. The API can be used on a computer that has

Mailbox server role installed, or a computer that has the Administrator-only install. CHKSGFILES is

available only as a 64-bit DLL in Exchange 2010. By default, the CHKSGFILES DLL is installed in the

C:\Program Files\Exchange 2010\bin directory.

The backup/restore application may choose to validate the integrity of a shadow copy backup set.

During a backup operation managed by the Volume Shadow Copy Service, there is no opportunity for

Microsoft Exchange Server 2010 to read each database file in its entirety and verify its checksum

integrity. Therefore, database and transaction log file integrity should be verified by the backup

application. Microsoft recommends that the backup application verify the physical consistency of the
http://msdn.microsoft.com/en-us/library/exchange/aa579076(v=exchg.140).aspxhttp://msdn.microsoft.com/en-us/library/exchange/aa579076(v=exchg.140).aspxhttp://msdn.microsoft.com/en-us/library/exchange/aa579076(v=exchg.140).aspxhttp://msdn.microsoft.com/en-us/library/exchange/aa579076(v=exchg.140).aspx


5/12


5

shadow copy set prior to informing the Exchange Writer that the backup is complete. Upon successful

backup, Exchange store updates the headers of the backed-up databases to reflect the last successful

backup times and removes transaction logs from the server that are no longer required to roll forward

from the last successful backup.

In order to provide better granularity and control during the consistency check process, Exchange 2010includes a set of unmanaged public APIs packaged into a DLL (chksgfiles.dll). The CHKSGFILES DLL is

installed with all Exchange server roles as well as part of the Admin-only option to enable running

consistency checks off of an Exchange server on a proxy backup machine.

More Reference:

http://msdn.microsoft.com/en-us/library/bb204049.aspx

http://msdn.microsoft.com/en-us/library/bb204219.aspx

5. Important Tool for Troubleshooting Exchange Backup issues

MS EventViewer

Microsoft Event Viewer (application events) is an important tool to be used for troubleshooting an

Exchange Backup. All the events will be recorded in sequencefrom the Backup initiation till the end.

Sequence of the Exchange Backup can be understood by looking at the application event during an

Exchange VSS backup. Provided below is a sample set of application events, from a backup taken from

the replica, and it is important to understand this sequence for troubleshooting purpose:

Replica Events:

SNo TIME SOURCE EVENTID

TASKCategory DESCRIPTION

1 3:27:34 AM MSExchangeRepl 2021 Exchange

VSS Writer

The Microsoft Exchange VSS Writer has successfully collected the

metadata document in preparation for backup.


VSS Writer

The Microsoft Exchange VSS Writer instance cfda266f-d071-47fd-9295-

8ec31213f378 has successfully prepared for a full or a copy backup of

database 'FRANCISTEST'. The following database will be backed up:

FRANCISTEST.


VSS Writer

The Microsoft Exchange Replication service VSS Writer (Instance cfda266f-

d071-47fd-9295-8ec31213f378) successfully prepared for backup.

4 3:27:49 AM MSExchangeIS

Public Store

9539 General General "The Microsoft Exchange Information Store database

"""" was stopped.

5 3:27:49 AM ESE BACKUP 960 General msexchangerepl (2444) This computer is performing a surrogate backup.

The master server is USSDWIN2K8R1.


VSS Writer


8ec31213f378 has successfully frozen the databases.


VSS Writer


d071-47fd-9295-8ec31213f378) successfully prepared for a snapshot.


VSS Writer


8ec31213f378 has successfully frozen the databases.
http://msdn.microsoft.com/en-us/library/bb204049.aspxhttp://msdn.microsoft.com/en-us/library/bb204049.aspxhttp://msdn.microsoft.com/en-us/library/bb204219.aspxhttp://msdn.microsoft.com/en-us/library/bb204219.aspxhttp://msdn.microsoft.com/en-us/library/bb204219.aspxhttp://msdn.microsoft.com/en-us/library/bb204049.aspx


6/12


6

9 3:27:49 AM MSExchangeRepl 2029

Exchange

VSS Writer


8ec31213f378 has successfully thawed the databases.


VSS Writer


d071-47fd-9295-8ec31213f378) has successfully processed the post-

snapshot event.

11 3:27:54 AM Storage Group

Consistency

Check

100 Initialization Instance 1: A physical consistency check has been successfully initiated.

Database

'\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy2\Program

Files\Microsoft\Exchange

Server\V14\Mailbox\FRANCISTEST\FRANCISTEST.edb' and the transaction

log files in

'\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy2\Program

Files\Microsoft\Exchange Server\V14\Mailbox\FRANCISTEST\' with a

base name of 'E05' will be validated.


Consistency

Check

200 Database

Header

Validation

Instance 1: Database headers have been successfully validated. All

databases are in a dirty shutdown state. To bring these databases to a

clean shutdown state, log generations 3 (0x00000003) to 3 (0x00000003)

will be required.


ConsistencyCheck

300 Log File

Validation

Instance 1: Transaction log files in '\\?

\GLOBALROOT\Device\HarddiskVolumeShadowCopy2\ProgramFiles\Microsoft\Exchange Server\V14\Mailbox\FRANCISTEST\' with a base

name of 'E05' have been

successfully validated. No errors were detected.

14 3:27:57 AM ESE 103 General "caagstart (788) Instance 1: Log File Integrity Check

(\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy2\Program

Files\Microsoft\Exchange Server\V14\Mailbox\FRANCISTEST\E05): The

database engine stopped the instance (0).

Internal Timing Sequence: [1] 0.000, [2] 0.000, [3] 0.000, [4] 0.000, [5]

0.000, [6] 0.000, [7] 0.000, [8] 0.000, [9] 0.000, [10] 0.000."




database engine started a new instance (0). (Time=0 seconds)

Internal Timing Sequence: [1] 0.000, [2] 0.000, [3] 0.000, [4] 0.000, [5]

0.000, [6] 0.000, [7] 0.000, [8] 0.000, [9] 0.000, [10] 0.000, [11] 0.000.




database engine (14.00.0692.0000) is starting a new instance (0).


Consistency

Check

400 Termination Instance 1: The physical consistency check has completed successfully. All

files were validated and no errors were detected.


VSS Writer

The Microsoft Exchange VSS Writer has successfully collected the

metadata document in preparation for backup.

19 3:29:03 AM ESE BACKUP 963 General msexchangerepl (2444) The surrogate backup procedure to

USSDWIN2K8R1 has been successfully completed.


VSS Writer

"The Microsoft Exchange Replication service VSS Writer instance

cfda266f-d071-47fd-9295-8ec31213f378 has successfully completed the

backup of database 'FRANCISTEST'.

Database log truncation has been requested for this database. Log

truncation will occur on the active copy after the next log generation is

created. Log truncation will occur automatically on the passive copies

after that log file is copied."

21 3:29:03 AM MSExchangeRepl 2033 Exchange The Microsoft Exchange Replication service VSS Writer (Instance cfda266f-


7/12


7

VSS Writer d071-47fd-9295-8ec31213f378) has successfully processed the backup

completion event.


VSS Writer


d071-47fd-9295-8ec31213f378) backup has been successfully shut down.

Active Events:

SNo TIME SOURCE EVENT ID TASK Category DESCRIPTION

1 3:27:51 AM ESE 210 Logging/Recovery "Information Store (3400)

FRANCISTEST: A full backup is

starting.

2 3:27:51 AM ESE BACKUP 910 General Information Store (3400) A full

surrogate backup is starting. This

server is the master. The requestor is

USSDWIN2K8R1.

3 3:29:04 AM MSExchangeRepl 3156 Service Active Manager successfully mounted

database FRANCISTEST on server

ussdwin2k8r1.gotir.ca.com.

4 3:29:04 AM ESE BACKUP 913 General Information Store (3400) The

surrogate backup procedure has

been successfully completed by

USSDWIN2K8R1.

5 3:29:04 AM ESE 213 Logging/Recovery "Information Store (3400)

FRANCISTEST: The backup procedure

has been successfully completed.

Will the transaction logs be purged from the Active or the Replicawhen an Exchange backup is

performed via the Replica?

The log truncation will happen on the Active Database only and not from the Copy databases.

Once the backup is completed from the Replica, it will signal the Active database. The logs will be

truncated from the Active database and sync will take place with the copies, which will affect the

transaction logs on the Copy databases.

Backup initiated transaction log file truncation will be triggered based on the type of backup chosen. In

non-DAG configurations, the Store Writer will truncate the transaction log files at the completion of

successful Full or Incremental backups. In DAG replicated configurations, log truncation will be delayed

by the Replication service until all necessary log files are replayed into all other copies. The Replication

service will delete the backed up log files both from the active and the copy log file paths after it verifies

that the to-be-deleted log files have successfully been applied to the copy database and both active

database and the database copies checkpoint has passed the log files to be deleted.

Log truncation doesn't occur on the active mailbox database copy when one or more passive copies are

suspended. If your planned maintenance activities are going to take an extended period of time (for

example, several days), you may have considerable log file buildup. To prevent the log drive from filling


8/12


8

up with transaction logs, you can remove the affected passive database copy instead of suspending it.

When the planned maintenance is completed, you can re-add the passive database copy.

6. User Problems related to Backup

6.1 User Problem 1 -Exchange Database Level Backup failsFrom the Application event, check at which step the backup failed.

Run a Windows Server Backup on that mailbox database to narrow down the case.

Refer the link below on how to run a VSS backup of Exchange using Windows Server Backup:

http://technet.microsoft.com/en-us/library/dd876854(v=exchg.141).aspx

6.2 User Problem 2 -Exchange Database Level Backup fails - Client halt error

The following errors will be observed from the Application event and it will occur during the backup

(after sequence 18 from the Application Events in section 5)

Active Events:Event ID: 215Source: ESECategory: Logging/RecoveryLevel: ErrorMessage: "Information Store (3084) EXCHDB1_08: The backup has been stopped because it was halted by the clientor the connection with the client failed.

Replica Events:

Source: ESE BACKUPEvent ID: 914Category: GeneralLevel: WarningMessage: General Information Store (3084) The surrogate backup by VMEXCHANGEMBS1A hasstopped with error 0xFFFFFFFF.

In this case as well, run a Windows Server Backup of Exchange, as explained above to narrow down the

case. If the Windows Server backup is successful, then do the steps below to disable TCP Offloading

TCP Offloading has to be disabled on the NIC from the Backup Server and the Exchange Nodes.

On the Backup servers:

IPv4 Checksum Offload Off

IPv4 Large Send Offload Disabled

IPv6 Checksum Offload Off

IPv6 Large Send Offload Disabled
http://technet.microsoft.com/en-us/library/dd876854(v=exchg.141).aspxhttp://technet.microsoft.com/en-us/library/dd876854(v=exchg.141).aspxhttp://technet.microsoft.com/en-us/library/dd876854(v=exchg.141).aspx


9/12


9

Receive Side Scaling Disabled

On the Exchange Nodes:

IPv4 Checksum Offload Disabled

IPv4 Giant TSO Offload Disabled

IPv4 TSO Offload Disabled

IPv4 TCP Segmentation Offload Disabled

Offload IP Options Disabled

TCP Checksum Offload (IPv4) Disabled

TCP Checksum Offload (IPv6) Disabled

6.3 User Problem 3 - Transaction Logs not truncated

Truncation of Transaction logs takes place after a full or an incremental backup of the Exchange

Database. If there is an issue where the transaction logs are not getting truncated, then the step is to

run a Windows Server backup of the Exchange Database to check if the logs are getting truncated.

If the transaction log truncation does not take place from the Windows Server Backup of the Exchange,

then check the status of the copy databases, in case of an Exchange 2010 DAG.

Log truncation doesn't occur on the active mailbox database copy when one or more passive copies aresuspended. If your planned maintenance activities are going to take an extended period of time (for

example, several days), you may have considerable log file buildup. To prevent the log drive from filling

up with transaction logs, you can remove the affected passive database copy instead of suspending it.

When the planned maintenance is completed, you can re-add the passive database copy.


10/12


10

6.4 User Problem 4 Unable to find Exchange Database

When the backup is initiated, you will get an error as E3577 Cannot find DB on Exchange server

In such cases, run the adsiedit.msc and use the same domain controller and credentials as provided in

the ARCserve Backup UI. Mostly, there will be ldap connectivity issues, or the Exchange containerscannot be seen. The solution is to use a different domain controller or credentials, so that the Exchange

objects can be viewed correctly.

A sample screenshot of the adsiedit.msc is provided below:

7. Exchange Restore

During a restore, the backup/restore application instructs the Exchange Writer to coordinate with the

Exchange Store to dismount the database, replace the database files, mount the database and then

replay the transaction logs as needed. When restoring, the requestor also communicates with the VSS to

prepare the system for the restore, and then to put the data back on to the mass storage device. The

backup/restore application is also responsible for working with the Windows Server to read data from

and write data to the backup storage media, whether a tape archive, a storage area network, or the

other backup medium. The restored database can be mounted either as a regular, active database, or as

the Exchange 2010 Recovery Database. Only one mounted database can be designated as a recovery

database on each Exchange Server.


11/12


11

8. Restore Sequence Diagram only Store Writer is used and not Replica

Writer

Ref:http://msdn.microsoft.com/en-us/library/exchange/bb204079(v=exchg.140).aspx

9. User Problems related to Restore

9.1 User Problem 1-Restore using Recovery Database

How to extract the Mailbox

After the mailbox database is restored to the Recovery Database, ensure that it is mounted.

Mount the Recovery Database either from Exchange Management Console or from Exchange

Management Shell by running the command Mount-Database "Recovery Database"

Make sure the destination mailbox is enabled and existing in the Exchange Server.

Make sure the database corresponding to the destination mailbox is mounted.

Before proceeding to recover the mailbox contents from Recovery Database, run the following

command to get the details of mailboxes available in the Recovery Database.

Get-MailboxStatistics Database "Recovery Database"
http://msdn.microsoft.com/en-us/library/exchange/bb204079(v=exchg.140).aspxhttp://msdn.microsoft.com/en-us/library/exchange/bb204079(v=exchg.140).aspxhttp://msdn.microsoft.com/en-us/library/exchange/bb204079(v=exchg.140).aspxhttp://msdn.microsoft.com/en-us/library/exchange/bb204079(v=exchg.140).aspx


12/12


12

The above command can be run to make sure that the mailbox that we are trying to restore is existing in the

Recovery Database.

Restore a single mailbox

1. Run the following command to restore the content of mailbox 'production mailbox' in the Recovery

Database to the production mailbox:

Restore-Mailbox Identity "production mailbox" RecoveryDatabase "Recovery Database"

2. Say 'Yes' for the confirmation asked. Restore process will start immediately.

3. When completed, you will get statistics about the restore mailbox job. You can see the information about

the source and target mailbox database etc.

Restore a single mailbox to a different mailbox

1. Run the following command to restore the content of 'production mailbox' in the Recovery Database to

'test mailbox' under the folder 'Recovered Contents Folder' within 'spare mailbox':

Restore-Mailbox -Identity "test mailbox" -RecoveryDatabase "Recovery Database" -

RecoveryMailbox "production mailbox" -TargetFolder "Recovered Contents Folder"

NOTE : The above command will create a new folder "Recovered Contents Folder" inside "test mailbox"

and will restore the content of "production mailbox" into the new folder "Recovered Contents Folder"

2. Say 'Yes' for the confirmation asked. Restore process will start immediately.

3. When completed, you will get statistics about the restore mailbox job. You can see the information about

the source and target mailbox database etc.

http://msexchangeguru.com/2010/05/23/exchange2010-recovery-database/

http://technet.microsoft.com/en-us/library/ee332351(v=exchg.141).aspx

9.2 User Problem 2 Unable to find the Recovery Database

When a restore to a Recovery Database is attempted, it will fail with unable to find the Recovery

Database. In order to narrow down the problem do the following:

Have the adsiedit.msc installed. Then connect to the same Domain Controller and check if the

containers can be displayed, and check if the Recovery Database created can be viewed from the

Active Directory (Refer 6.4 User Problem 4 for a screenshot of the adsiedit.msc)
http://msexchangeguru.com/2010/05/23/exchange2010-recovery-database/http://msexchangeguru.com/2010/05/23/exchange2010-recovery-database/http://technet.microsoft.com/en-us/library/ee332351(v=exchg.141).aspxhttp://technet.microsoft.com/en-us/library/ee332351(v=exchg.141).aspxhttp://technet.microsoft.com/en-us/library/ee332351(v=exchg.141).aspxhttp://msexchangeguru.com/2010/05/23/exchange2010-recovery-database/

backup and recovery exchange 2010 dag

Documents