0 Copyright Fujitsu, Release February 2012
ETERNUS DX Advanced Copy Functions
REC (Remote Equivalent Copy)
REC ETERNUS DX Advanced Copy Functions
Content
REC Overview REC Modes Synchronous Split and Recovery Sub-modes Asynchronous Transmission Sub-modes REC in Detail Differences Between REC Modes Skip Initial Copy Combinations of Advanced Copy
REC ETERNUS DX Advanced Copy Functions Copyright Fujitsu, Release February 2012 1
REC Overview
REC Usage Business Continuity: Disaster Recovery REC Concept Diagram REC Suspend and Resume REC Process REC Concurrent Suspend
REC ETERNUS DX Advanced Copy Functions Copyright Fujitsu, Release February 2012 2
Copyright Fujitsu, Release February 2012
REC Usage (1) Remote copy between two ETERNUS systems High Availability Short distances
• Data transfer can be achieved using FC or iSCSI Separate, independent interconnection lines between the ETERNUS
systems are necessary
REC ETERNUS DX Advanced Copy Functions
SAN
Operation Volume
Brocade Fiber Channel Switch
Site B
ETERNUS DX
Management Server
SAN
Operation Volume
Brocade Fiber Channel Switch
Site A
ETERNUS DX
Management Server
3
REC Usage (2) Remote copy with ETERNUS systems Disaster Recovery Very long distances
• 10+ km data transfer can be achieved by FC-WAN converter or iSCSI Separate, independent interconnection lines between the ETERNUS
systems are necessary
Copyright Fujitsu, Release February 2012 REC ETERNUS DX Advanced Copy Functions
Management Server
SAN
Advanced Copy Function REC
Operation Volume
Main site
ETERNUS DX
WAN/LAN SAN
Backup Volume
Management Server
ETERNUS DX
Backup site
Brocade Fiber Channel Switch
Management Server
Brocade Fiber Channel Switch
4
Business Continuity: Disaster Recovery Recovery Point Objective, RPO How much data has been lost?
Recovery Time Objective, RTO How soon after the event is the application available again?
Copyright Fujitsu, Release February 2012 REC ETERNUS DX Advanced Copy Functions
RTO RPO
Time
Last known good image
Application is available again
Application Downtime
Recover
Yesterday 24 hrs Now
All modifications that are lost
Analyze
Restore
5
REC Concept Diagram Background copy transfers sequentially all data of the Source
Volume to the target site While also updating both the Source and Target with any new write data
REC must be started and reach the equivalent state before the Destination Volume can be detached (suspend or cancel) The Destination Volume is again writeable after suspend or cancel
Copyright Fujitsu, Release February 2012 REC ETERNUS DX Advanced Copy Functions
Source at
site A
Target at
site B
Start REC Write
Background Copy Paired
Initial Copy complete
Mirroring
Write Stop REC (Split)
I/O Req.
After Split, both Volumes can be
accessed independently
I/O Req.
6
Cancelling REC terminates the session and a complete re-mirroring will be required when a new REC session is started All blocks of the Source need to be copied After resuming from Suspend state only the changes made
during the suspended state are copied Changes to both the Source and Destination Volumes are recorded in a
bitmap during the suspended state
REC Suspend and Resume
Copyright Fujitsu, Release February 2012 REC ETERNUS DX Advanced Copy Functions
Suspend REC
Re-mirroring
Equivalent state
S D
S D
Cancel REC Write to source
Copy All data
Using the bitmap information,
copy changed data only!
Start REC
Resume S D
Equivalent state
Changes to Source are recorded when suspended
or destination volume
and Destination volumes or overwrite data only!
7
REC Process REC is started Replication proceeds in the background
• Block level copy from the local Source to the remote Destination Copy rate depends on ETERNUS priority setting (high, medium or low)
• Parameter to be assigned via the ETERNUS GUI or CLI The Source Volume remains available The Destination Volume is inaccessible Throughput is degraded during initial copy
Replication is complete The mirrors are kept in Equivalent state
Detach request (Cancel or Suspend) Must be coordinated with the application to obtain a logically "consistent"
copy • For example, flush system buffer, shutdown application, etc.
Both mirror Volumes are available for write access while suspended or after cancelling
Copyright Fujitsu, Release February 2012 REC ETERNUS DX Advanced Copy Functions 8
REC Concurrent Suspend Concurrent Suspend function This allows consistent copies of multi-volume data objects, such as
databases All REC replication pairs that are involved have to be grouped Possible after REC session has reached Equivalent state Multiple REC sessions can be suspended at the same point in time
Copyright Fujitsu, Release February 2012 REC ETERNUS DX Advanced Copy Functions 9
REC Modes
REC Synchronous Modes REC Asynchronous Modes
REC ETERNUS DX Advanced Copy Functions Copyright Fujitsu, Release February 2012 10
REC Synchronous Modes Write confirmation is responded to the host after data transfer
to the target ETERNUS DX is completed This mode guarantees that the copy process is fully completed
on both sides of the REC before an acknowledgement of a data write request is returned to the server
Copyright Fujitsu, Release February 2012 REC ETERNUS DX Advanced Copy Functions
Source Volume Site A
Target Volume Site B
Write Acknowledged
Application
Copy
OK
11
REC Asynchronous Modes Write confirmation is responded to the host after data write to
local cache of the ETERNUS DX system This mode will not guarantee that the copy process is fully
completed on both sides of the REC before any update or write process response is returned to the server
Copyright Fujitsu, Release February 2012 REC ETERNUS DX Advanced Copy Functions
Queue
Write Acknowledged Acknowledged Write
Source Volume Site A
Target Volume Site B
Application Application
12
Synchronous Split and Recovery Sub-modes
Transfer Mode "Automatic Split" Transfer Mode "Manual Split" Line error options/recovery with "Automatic Recovery" or
"Manual Recovery"
REC ETERNUS DX Advanced Copy Functions Copyright Fujitsu, Release February 2012 13
Transfer Mode "Automatic Split" During path failure Automatic disconnection of the REC session might occur
• For example if the transmission fails during the REC process Write access to Source Volume can continue even when the REC
session is in Hardware Suspend • Changes are tracked by the bitmap mechanism
Server continues to write on the copy Source only The source and target data are no longer in Equivalent state
• Source and Destination LUNs need to be synchronized after the connection problem has been solved
Copyright Fujitsu, Release February 2012 REC ETERNUS DX Advanced Copy Functions
Source
Target
Write Write Write Write
Copy Copy Copy
Time
Write
Automatic Split mode
Not Copied Failure
14
Transfer Mode "Manual Split During path failure The REC session stops completely and all writes are suspended Source and Target Volumes remain in Equivalent state
• If the path failure is a result of a site disaster this prevents data losses but on the other hand means unscheduled downtime for the applications
To regain access to the Source LUN of the ETERNUS, "Suspend" needs to be executed
Copyright Fujitsu, Release February 2012 REC ETERNUS DX Advanced Copy Functions
Source
Target
Write Write Write Write
Copy Copy Copy
Time
Write
Not Copied Failure
Manual Split mode
15
Line Error Options/Recovery Automatic Recovery REC session is automatically resumed after the data path functionality
has been recovered
Manual Recovery REC session must be manually "Resumed" after the data path is restored
• Before issuing "Resume", this option allows clone creation (OPC, EC) of the mirror Volumes • Ensures that the last known good data images are preserved before writing on the
Volumes is re-enabled
As both mode options, Manual Split and Manual Recovery, require manual or scripted intervention, they are recommended for experienced users only
Copyright Fujitsu, Release February 2012 REC ETERNUS DX Advanced Copy Functions 16
Asynchronous Transmission Sub-modes
Stack Mode Example of Using Stack Mode Asynchronous Mode - Consistency REC Buffer Transfer Parameters Consistency Mode Buffer Configuration Consistency Mode Usage
REC ETERNUS DX Advanced Copy Functions Copyright Fujitsu, Release February 2012 17
Asynchronous - Stack Mode (1) This mode uses a bitmap Only updated block locations are recorded in the bitmap before sending
acknowledgement to the server • Data itself is not cached
Transfer of the changed blocks is performed by an independent transfer engine that scans the bitmap Write order is NOT maintained in the
destination Useful when limited bandwidth
is available but a large amount of data needs to be transferred
Copyright Fujitsu, Release February 2012 REC ETERNUS DX Advanced Copy Functions
1. Write data to disk 2. Record updated block 3. Write acknowledgement
Write
...
Write Ack.
Write I/O
Bit map
1 1
18
Asynchronous - Stack Mode (2) When multiple writes have occurred in the same block
address, only the latest data at the time of the next patrol check is sent to the copy destination, helping to minimize the amount of transferred data This mode can lead to a loss of
data if a malfunction occurs in the source system This mode is mainly used for
backing up data to a remote system
Copyright Fujitsu, Release February 2012 REC ETERNUS DX Advanced Copy Functions
...
Transfer Engine Process
Transfer Engine
4. Transfer engine search and find an updated block
5. Transfer updated block
1
1
1. Write 2. Write
19
Example of Using Stack Mode (1) Example: REC Multi Copy Two REC (Stack) sessions are set up
and alternately operate as Suspend / Active (2) Example: REC Cascade Copy Two Local Copy sessions are set up
and operate at the REC (Stack) copy destination
Creating a Snapshot to a Remote site (remote backup)
When a data failure occurs at source site • Restore is performed using the session volume
that was in the suspend state at that time Suspending a session not possible
in Stack mode, only in Async. mode Stack mode -> Force suspend Switch to through mode Resume -> Suspend
Copyright Fujitsu, Release February 2012 REC ETERNUS DX Advanced Copy Functions
1
2
20
Asynchronous Mode - Consistency (1) This mode works always with REC Buffer Cache Receive buffer and send buffer as a pair The write order across multiple REC sessions can be
guaranteed at the copy destination i.e. write consistency over multiple LUNs or regions Updates to multiple LUNs on the ETERNUS are accumulated
in the REC buffer and then grouped in buffer units for transfer This reduces the communication sequences with the remote site and
enables more efficient use of line capabilities (increased performance)
Copyright Fujitsu, Release February 2012 REC ETERNUS DX Advanced Copy Functions 21
Asynchronous Mode - Consistency (2) This mode uses ETERNUS REC buffers to store actual data Not only bitmap pointers
REC buffers are configured as either Receive or Transmit At check point, all access to
Source Volumes are stopped Less than 100 micro seconds Overhead is less than 0.1%
Check point interval can be set between 1 and 120 seconds This has an affect to the RPO
Buffer data is applied to the remote site only after the entire buffer is received
Copyright Fujitsu, Release February 2012 REC ETERNUS DX Advanced Copy Functions
Write data copies to buffer
Check Point
Source
Destination
Buffer switch, then transfer
Transfer complete, then copy to dest.
Send Buffer
Receive Buffer
22
Asynchronous Mode - Consistency (3) In this mode a large amount of data accumulates in the REC
buffer All the accumulated data is transferred to the destination as a
block at user definable intervals There are three REC buffer parameters for the asynchronous
consistency mode that are set with the ETERNUS Web GUI
Copyright Fujitsu, Release February 2012 REC ETERNUS DX Advanced Copy Functions
Item Description Forwarding Interval (sec) Specify the data forwarding interval
Watch (Monitor) Time (minutes)
Specify the monitoring time before transition changes to Halt status for a REC session when a REC buffer shortage occurs. If the REC buffer remains in high-load state (overload) for the specified time, the status of REC session will be automatically changed to Halt. When the Watch Time is set to 0, the REC buffer is not monitored.
Halt Wait Timer (seconds) Specify the maximal non-response time. The ETERNUS monitors the host I/O waiting time. During this time the host I/O is stopped from ETERNUS point of view. This gives priority to perform REC buffer transfers. If the waiting time exceeds the specified time, REC session status automatically changes to Halt status.
23
REC Buffer Transfer Parameters Buffers can be set to either Send or Receive with following
setup Send Buffer at REC source Receive Buffer at REC destination Up to eight1) buffers (in total) can be set per system In the REC asynchronous Consistency mode, there may be Eight connected ETERNUS DX pairs Each pair has one connection channel with a send and receive buffer This only allows copy in one direction
• But the direction can be reversed from the command line by changing the REC buffer role (swsrpbuffset)
For model specific configuration details please refer to the Remote Advanced Copy Function Guide Many ETERNUS DX models provide also REC Disk Buffer
functionality
Copyright Fujitsu, Release February 2012 REC ETERNUS DX Advanced Copy Functions 24
1) Four with DX90 and DX90 S2, please refer to product specification for up-to-date information.
Consistency Mode Buffer Configuration Multiple pairs of REC buffers have been configured to enable
copying between multiple systems (1) Enables copying between multiple locations (1) Use single buffer pair for unidirectional data transfer only (2) Enables copying in both directions between systems (2) Use two buffer pairs for bidirectional data transfer
• For forward and reverse data transfer NOTE: It is not possible to configure 2 x send buffers addressing the
same destination ETERNUS system
Copyright Fujitsu, Release February 2012 REC ETERNUS DX Advanced Copy Functions
Receive REC buffer
Site A
SEND REC buffer
SEND REC buffer
Receive REC buffer
Site B
Pair 1
Pair 2 SEND REC buffer
Site A
SEND REC buffer
Receive REC buffer
Site B
Receive REC buffer
Site C
Pair 2
1 2 Pair 1
25
Consistency Mode Usage (1) Consistency pairs can be
established between two systems and for each transfer direction Multiple applications can
share one consistency pair Each consistency pairs can
be individually controlled Backups can be made
separately for each system Restore can be used for recovery, independently of the consistency pair
Copyright Fujitsu, Release February 2012 REC ETERNUS DX Advanced Copy Functions
Site A Site B
App-Sys A
App-Sys B
App-Sys A
App-Sys B
REC buffer
send
REC buffer
receive
REC buffer
receive
REC buffer
send
App2-Sys C App1-Sys C
App2-Sys C App1-Sys C
Consistency Pair 1
Consistency Pair 2
26
Consistency Mode Usage (2) REC Consistency mode preserves write order During the copy process the updated blocks are stored in different sets of
REC buffers located in the CM cache All REC buffer sets are transmitted in one transaction
• Either all buffer sets are copied or nothing is copied
The receiving ETERNUS system copies the data to the disk only after having received the complete buffer contents
The transmitting ETERNUS system will not discard buffer contents until the receiving ETERNUS system acknowledges the reception
Copyright Fujitsu, Release February 2012 REC ETERNUS DX Advanced Copy Functions 27
REC in Detail
Asynchronous Mode Suspend and Resume Asynchronous Mode Line Failure Asynchronous Through Mode
REC ETERNUS DX Advanced Copy Functions Copyright Fujitsu, Release February 2012 28
Asynchronous Mode - Suspend/Resume (1) During Suspended state, modified blocks are marked at each
side in bitmap tables Marked blocks at target side are overwritten (nullified) after
resume
Copyright Fujitsu, Release February 2012 REC ETERNUS DX Advanced Copy Functions
Source Volume Site A
Target Volume Site B
A*
Write
Bitmap Table
REC Re-Sync Copy
Keep track of modifications
REC Maintain Copy
A
A
REC Maintain Copy
A**
A**
A*
Suspend
C
Bitmap Table
Resume
Merge Bitmap Table
A*
1
1
Write
Write Write
Overwrite
29
Asynchronous Mode - Suspend/Resume (2) This picture illustrates the bitmap merge after Resume is
invoked
Copyright Fujitsu, Release February 2012 REC ETERNUS DX Advanced Copy Functions
Suspend Source Destination
Cache
Bitmap
Cache
Bitmap
Source physical
disk
Destination physical
disk
Resume Source Destination
Cache
Bitmap
Cache
Bitmap
Source physical
disk
Destination physical
disk
Bitmap merge
30
Asynchronous Mode - Line Failure During path failure, the modified blocks are marked in
bitmap table After path becomes available again, the marked blocks are
transferred without preserving the write order
Copyright Fujitsu, Release February 2012 REC ETERNUS DX Advanced Copy Functions
A*
Write
REC Re-Sync
Copy
REC Maintain
Copy
A
A
Write
REC Maintain
Copy
A**
A**
A*
Halt State
First Write after path failure returns sense information
Re-synchronization
A*
Path Failure Path Available
Keep track of modifications
Source Volume Site A
Target Volume Site B
Write
Bitmap Table
31
REC Asynchronous Through Mode This mode is a special asynchronous mode It is used to flush as-yet-untransferred data when a Stack or Consistency
mode session is in • Stop state (due to Cancel command) • Suspend state
This mode should be used when an interconnect path failure occurs after a Cancel or Suspend is initiated After the path is restored the buffered data must be flushed using the
REC Asynchronous Through Mode
Copyright Fujitsu, Release February 2012 REC ETERNUS DX Advanced Copy Functions 32
Through Mode for Sequential Writing This diagram shows the impact to the server I/O when running
"Asynchronous Through Mode" The performance of this mode equals that of the REC
Synchronous Mode For these reasons, you should not use the REC Asynchronous Through
Mode under normal circumstances
Copyright Fujitsu, Release February 2012 REC ETERNUS DX Advanced Copy Functions
Server ETERNUS DX Copy Source
ETERNUS DX Copy Destination
1. Update data
1. Acknowledge 2. Update data
2. Acknowledge
ETERNUS
1. Acknowledge
Source ETERNUS DX has to wait for the acknowledgement of the previous data transfer.
33
Differences Between REC Modes
Network Bandwidth Differences Between Modes REC Modes of Operation - Summary
REC ETERNUS DX Advanced Copy Functions Copyright Fujitsu, Release February 2012 34
Network Bandwidth Synchronous and Consistency modes: Match to peak times
Stack mode: Match to average load
Copyright Fujitsu, Release February 2012 REC ETERNUS DX Advanced Copy Functions
Update quantity
Required bandwidth
Update quantity
Required bandwidth
35
Differences Between Modes Transfer mode comparison
Copyright Fujitsu, Release February 2012 REC ETERNUS DX Advanced Copy Functions 36
REC Modes of Operation - Summary Synchronous REC Application waits until update reflected on local and remote volume Order guaranteed No data loss High impact on performance Automatic or manual split options Asynchronous REC (Stack mode) Application waits until update reflected locally and placed in move queue Independent process sends updates to destination No update order guarantee Data loss possible Low impact on performance level Asynchronous REC (Consistency mode) Application waits until update placed into local group send queue After timer interval (1, 2 or 4 seconds) transmitted to remote receive
queue Update order guaranteed Data loss possible Low impact on performance level
Copyright Fujitsu, Release February 2012 REC ETERNUS DX Advanced Copy Functions 37
Skip Initial Copy
Start+Suspend / Resume+Remain Initial Copy Skip Mode
REC ETERNUS DX Advanced Copy Functions Copyright Fujitsu, Release February 2012 38
There is a special ACM command option available to support this feature for REC only Start+Suspend Resume+Remain
Start+Suspend / Resume+Remain
Copyright Fujitsu, Release February 2012 REC ETERNUS DX Advanced Copy Functions
Suspend
Write
EC Maintain
Copy
Write
Source Volume Site A
Target Volume Site B
Start +
Suspend
Tape Backup
Tape Restore
Operation can be continued
Suspend
Resume +
Remain
Time line
39
When the Start+Suspend is issued the Copy Source data has to be consistent from the application point of view The Copy Destination data has to be 100% identical with the
Source data when the Resume+Remain is issued Note, ETERNUS does not verify the data consistency between the two
Volumes After the synchronization of the Destination is completed, ETERNUS
shows always equivalent state for the Volumes • Regardless if the data actually is consistent
Initial Copy Skip Mode
Copyright Fujitsu, Release February 2012 REC ETERNUS DX Advanced Copy Functions 40
Combinations of Advanced Copy
Cascade Copy Multi Copy
REC ETERNUS DX Advanced Copy Functions Copyright Fujitsu, Release February 2012 41
Concatenates several copy sessions An existing copy Destination Volume becomes the copy
Source for a new copy session
Cascade Copy
Copyright Fujitsu, Release February 2012 REC ETERNUS DX Advanced Copy Functions
OPC, QuickOPC EC
REC - Stack
REC - Synchronous - Stack - Consistency
OPC, QuickOPC SnapOPC EC
REC - Stack - Consistency
REC - Stack
REC - Consistency
REC - Synchronous
Source-1 Source-2
Destin-2 Destin-1
Source-2 Destin-2 Destin-1
Source-1 Source-2
Destin-2 Destin-1
Source-1
ETERNUS DX-1 ETERNUS DX-2 ETERNUS DX-3
ETERNUS DX-1 ETERNUS DX-2 ETERNUS DX-3
42
Copies the same copy source area to multiple copy destinations The destination can not be the source for another multi copy
session
Multi Copy
Copyright Fujitsu, Release February 2012 REC ETERNUS DX Advanced Copy Functions
REC
ETERNUS DX-1
Destin-2
Destin-1
Source-1
ETERNUS DX-2
Destin-1
43