module 4 - data domain advanced features and functions_v5

1 EMC CONFIDENTIAL—INTERNAL USE ONLY.

Data Domain Advanced Features and Functions

Velocity Partner Accreditation


Module Objectives

Upon completion of this module, you will be able to:

• Describe VTL and VTL library planning

• Describe snapshots, fastcopy, and data retention

• Describe data replication and recovery

• Describe DD Boost and integration with EMC NetWorker

• Describe capacity and throughput planning

• Describe Data Domain system monitoring tools


Lesson: Virtual Tape Library (VTL)

Upon completion of this lesson, you will be able to:

• Describe a Data Domain VTL

• Describe VTL library planning


VTL Definition

Application/

media backup

server

Backup Cache

Retention/

restore/

cloning

Disaster recovery/

archive/

offsite storage

Why should you use a VTL?


Configuration Terms Barcode • Unique ID assigned to virtual tape when you create it

• In the Data Domain OS aka: label, tape label

CAP • Cartridge access port (CAP), emulated tape enter/eject point for moving tape to/from a library • In the Data Domain OS aka: mail slot

Library • Emulates physical tape library with tape drives, changer, CAPs, and slots (cartridge slots) • In the Data Domain OS aka: autoloader, tape silo, tape mount, tape jukebox, vault

Pool Collection of tapes that map to a directory on a file system, used to replicate tapes to a destination

Tapes • Represented in a system as files. You can export/import from a vault to a library, move within a library across drives, slots, and CAPs • In the Data Domain OS aka: cartridge

Vault Unused tapes stored in vault, tapes are in library/vault


VTL Library Planning

Fibre channel

• 256 virtual tape drives

(DD880 only)

• 128 virtual LTO-1, LTO-2,

LTO-3 tape drives

(all other models)

Robot loads\changes

tape cartridge

VTL

Up to 1,00,000

tape cartridges

in virtual vault

• 64 virtual libraries

• 20,000 slots per library

• 100 CAPs per library

• 1000 CAPs per system

• 800 GiB per tape


Capacity Planning

• More planning needed at installation

• Expired tapes NOT deleted, space not reclaimed till tape is overwritten/deleted

• Always create more slots that you need

• Load tapes when you need them

• Stop loading tapes once retention requirements are met


Lesson: Fastcopy, Snapshots, and Data Retention


• Describe Data Domain fastcopy

• Describe Data Domain snapshots

• Describe Data Domain data retention

• Explain the Data Domain cleaning process


Fastcopy

Copy

If you change source or target directory while copying, they will not be equal.

Source directory

Target directory


Snapshots

Original copy Snapshot copy

/data/ coll /backup /data/ coll /backup / .snapshot

/data/ coll /backup/files /data/ coll /backup/files/ .snapshot

Snapshot taken at 22:24 GMT 22:24 GMT snapshot saved


Retention Lock

Archive software

or user initiates

Prevents retention-locked files from being deleted/modified for up to 70 years

Licensed feature

Retention locked files can be stored, encrypted, and replicated


Retention Lock Flow

1. License/enable retention lock

2. Set min/max retention period 3. Create file

4. Lock file (set retention period)

- Extend retention-locked file

- Delete expired retention-locked file

5. Transfer file to Data Domain system


curr

ent

tim

e

Configure Client File Retention Period

minimum retention period valid atime period

maximum retention period

2. Data Domain system administrator sets min/max retention periods on Data Domain system

1. User creates file and sets last access time (atime) to desired retention period

Client must initiate retention lock

3. File either committed as a retention-locked file or ignored


File System Cleaning

File A deleted with no retention lock File A deleted at next cleaning

File B deleted, retention lock initiated File B maintained until retention lock period ends

SW backups to Data Domain

Cleaning reclaims physical storage occupied by deleted objects


Cleaning

Disk blocks

What?

Reclaim space

Disk block

Disk block

Why?

House keeping (reclaim “dead” segments)

Performance (rewrite duplicate data)

House keeping

Performance tuning

Container 1

Container 2

Container 2 Container 3

dead copy Forward valid

Container 1

or Free space

Free space


Lesson: Replication and Recovery


• Describe the types of Data Domain replication

• Identify how replication improves storage

• Describe the data recovery process


Data Replication

New deduplicated compressed data is automatically replicated to destination

WAN

LAN

Source Destination


Data Domain Replication Types

• Collection: for entire site backup

• Directory: for partial site backup

• Pool: for VTL files/tape backup


Data Domain Collection Replication

/backup /backup

Source Destination

• Immediate accessibility

• Read only

• User accounts/passwords

replicated from source

• Works with encrypted files

• Works with retention lock

Recovers

entire system


Data Domain Directory Replication

/backup/dir a

Source

• Destination must have available storage

• CIFS and NFS clients ok

• Do not mix CIFS/NFS data in same directory

• Destination directory created automatically

• Works with encryption

• Works with retention lock

/backup/dir b

/backup/dir a

/backup/dir b

Destination

Destination

Recovers selected data


Data Domain Pool Replication

Source Destination

• Works like directory replication

• Destination doesn’t require VTL license

pool 3

pool 1 pool 1

pool 2 pool 2 pool 3


Replication Context

WAN

Source Destination


Replication Topologies Source

Source

Source

Destination

Destination

Destination

Source

Source

Source

Destination

Destination

Destination

1 to 1

1 to many

Bi-directional

Many to 1

Cascaded Cascaded 1-to-many

Primary source/ Destination

Primary source/ destination


Recover Data

Backup server File server Clients

On site Off site disaster recovery

WAN

Replication

In case of disaster, recover off-site replica

You can configure a Data Domain system to store backup data and retain onsite for 30-90 days


Why Resynchronize Recovered Data?

WAN

Source

Resynchronization

Destination

Recreate deleted context Out of space

Convert collection to directory replication


Lesson: Data Domain Boost


• Describe DD Boost

• Describe replica awareness

• Describe how DD Boost works with EMC NetWorker

• Describe supported network topologies

• Describe DD Boost advanced load balancing and link failover feature


DD Boost • Provides standard/centralized management features

through backup software • Works with industry standard backup software

– EMC Networker – Symantec NetBackup (Data Domain plug-in required) – Symantec Backup Exec (Data Domain plug-in required)

• Enables advanced load balancing and failover • Requires licenses on Data Domain System

– DD Boost – Replication (if used) Note: Your backup software might require

license to enable the feature. Verify your backup software documents.


DD Boost (contd.)

Backup Server

OST plug-in

DD Boost

Clients

Clients send data to backup server

Less data sent over LAN

Deduplication/compression occur in backup server

LAN LAN

Optimized protocol for high throughput

Manages connections between backup applications and Data Domain systems with DD Boost

Deduped Data Stored


Replica Awareness

Backup Server

WAN

Backup site Disaster recovery site

OST plug-in

Initiates and tracks replication for easy management and disaster recovery

Archive to tape as needed

You manage replication from backup server console

replication DD

Boost DD

Boost


DD Boost Advantage • Without DD Boost

– Backup server(s) not aware of Data Domain replica(s)

– Recovery is manual process

• With DD Boost – Backup server dedupes data

and minimizes network bandwidth use

– Replication and recovery are centrally configured and monitored

Backup

Replication

Optimized deduplication

Replication engine

Backup server

DD Boost server

DD Boost server

Without OST

Manually configured replication

With DD Boost

OST plug-in


NetWorker – Work Flow

Start clone

(Clone 1)

4

NetWorker

Server

Control

Data

Local

Data Domain system

Remote

Data Domain system

5 Data

transfer

6 Done (Clone 1)

Clone 1

Save

Set 1

Clone 1

update

control

data

7

Clone 1

Save

Set 1

New data backup

(Save Set 1)

1

2 Done (Save Set 1)

Save Set 1

update

control data

3


Lesson: Capacity and Throughput Planning


• Describe capacity planning and its importance

• Describe throughput planning and its importance


Monitor File System Space Use • Factors that effect how fast data on disk grows

– Size of data sets getting backed up – Compressability of data getting backed up – Retention period specified in backup software

• Monitor disk use closely when you back up large data sets that show low compression factors and have large retention times

• You can get more accurate space-use view from CLI

• Use filesys show space to monitor post-compression data growth


Space Graph

Cumulative physical data

written to DDS Amount of Data within

Backup Application

Compression Ratio:

Pre-compression/ Data Collection

Available Space on DDS


Space Graph (contd.)

What does the saw-tooth line for compression ratio represent?


Compression Factor Calculation

Original bytes

Data Domain system data written

Compression factor

What does cleaning do to this equation?

It decreases the Data Domain system data written (denominator)

and thus increases the compression factor.


How much?

• Data size (TB)

• Data type

• Full backup size

• Compression rate

(deduplication)

Capacity Planning: Determine Capacity Needs

Capacity

needs

How long?

• Retention policy

(duration)

• Schedule


Determine Capacity Needs (contd.)

• Data Domain system internal indexes and other

components use variable storage amounts

depending on data type and file sizes

• If different data sets are sent to identical systems,

one system may, over time, have room for

more/less backup data than another

• Challenging data types – Pre-compressed (multimedia, .zip, and .tiff) – Encrypted


Compression Requirements with Variables

• 5x – Nearline and archive – Incremental + weekly full backup with two weeks retention

– Daily full backup with one week retention

– Nearline and archival use compression tends to be capped here

• 10x – Overall compression – Incremental + weekly full backup with one month of retention

– Daily full backup with two-three weeks retention

• 20x – Overall compression – Incremental + weekly full backup with two-three months retention

– Daily full backup with three-four weeks retention


Calculate Required Capacity

1st full backup Incremental

backup 4

Weekly full

backup

number of

weeks

Required

capacity

Total space required

Required

capacity


Calculate Required Throughput

Largest backup

Backup time window

6 TB

10 hrs Example

Required throughput

600 GB/hr


System Model Capacity and Performance

• Maximum capacity is amount of usable data storage space

• Maximum capacity based on max number of drives supported by a model

• Maximum throughput is achieved using either VTL interface and 4Gbps Fibre Channel or DD Boost and 10Gb Ethernet

• Current model throughput and capacity specifications http://www.datadomain.com/products/

http://www.datadomain.com/pdf/DataDomain-Product-Overview.pdf


Select Model

• Be conservative when determining which model to use • Use 75-85% of model capacity and throughput

(factor 15-25% buffer for capacity and throughput)

Required capacity

Maximum logical capacity

Required throughput

Maximum throughput

Capacity %

Throughput %

100

100


Calculate Capacity Buffer for Selected Models

Required capacity

Maximum capacity

DD140 example

840 GB

860 GB

840 GB

1650 GB

DD610 example

97%

51%

100%

100%

100%

% of Maximum capacity

3% Buffer not ok

51% Buffer ok


Match Required Capacity to Model Specifications

OR? 1,650 GB with 7 drives

DD610 DD140

860 GB

Ensure capacity buffer is big enough

For example

Required capacity = 840 GB


Calculate Performance Buffer for Selected Models

Required throughput

Maximum throughput

DD610 example

600 GB/hr

675 GB/hr

600 GB/hr

1126 GB/hr

DD630 example

89%

53%

100%

100%

100%

% of Maximum throughput

11% Buffer not ok

47% Buffer ok


Match Required Capacity to Model Specifications

OR? 1.1 GB/hr

DD630 DD 610

860 GB/hr

Ensure performance buffer is big enough

For example

Required throughput = 600 GB/hr


Lesson: System Monitoring Tools

Upon completion of this lesson, you will be able to: • Describe Data Domain system monitoring tools

– SNMP – syslog – autosupport – SUP


Alert

Monitoring a Data Domain System

Data Domain

system administrator

Daily

alerts and

autosupport reports

Daily

alerts and

autosupport reports

Data Domain

technical support

2. syslog 3. autosupport 4. SUB 1. SNMP


SNMP

• You can monitor a Data Domain system via SNMP utilities • You can integrate the Data Domain Management

Information Base (MIB) into SNMP monitoring


Syslog (Remote Logging)

-

syslog server

LAN Port 514

System messages

Port 514

Sends system messages to remote syslog server

Uses TCP port 514

You collect logs


Autosupport

• Easy to install – just once at system setup • Helps solve/prevent system problems

– Provides timely notification of significant issues – Enables rapid response time to address or prevent problems – Includes critical system data to aid support case triage and

management


Autosupport System autosupport@

autosupport.datadomain.com

System history

Integration to

other systems

Reports

Data Domain technical support

(support case)

Other vendors

Daily alert summary

Reboots

Summary autosupport report

Warnings

via SMTP

Detailed autosupport report


Autosupport Types

autosupport

types

Non scheduled Scheduled

Detailed

autosupport

report

sent 6 am

Daily alert

summary

email

sent 8 am

Alerts

• warning

• failure

Reboots


Autosupport Via Enterprise Manager • Data Domain systems provide alerts, autosupport reports, and logs • Access through Enterprise Manager


Autosupport Reports

• Using SMTP, sent to Data Domain technical support daily at 6 am local time (default) • Contains system ID, uptime information, system command outputs, runtime parameters, logs, system settings, status and performance data, and other debugging information • Long text report (500-800K) • Sections parsed into data warehouse for analysis and reporting

Subscribers receive daily detailed reports


Detailed Autosupport Report Contents


Daily Summary Autosupport

Provides summary autosupport report

Tells you if the system is ok or not. If not, no email receipt is received

Sent daily at 8 am

Uses autosupport email distribution list

This is an example of an alert


Alerts

Unique numerical ID

Physical component where alert occurred

Subsystem where alert occurred

Alert severity

Date and time alert occurred


Alerts Notification

Terse description of event

Sent immediately upon detection

Creates support case

Has separate email distribution list


Logs

Every Sunday at 3 am 1. New log file opened 2. Old log file renamed

CLI: log view filename


Support Upload Bundle (SUB)

• Large (multi-GB sized) tar file • Contains

– OS settings and log files – System files (not customer data files) identified as needed

for system diagnosis by Data Domain support and engineering

• Used to triage and diagnose a Data Domain system in the field • CLI commands used to generate and optionally send (via http)

SUB to Data Domain support site • Generated by sysadmin on Data Domain system via GUI/CLI


Module Summary

Key points covered in this module include:

• VTL and VTL library planning

• Snapshots, fastcopy, and data retention

• Data replication and recovery

• DD Boost and integration with EMC NetWorker

• Capacity and throughput planning

• Data Domain system monitoring tools


Product Demo

Click the link below to view a demonstration on Data Domain.

Launch Demo

Data Domain Advanced Features and Functions Demo.swf

PROPERTIES

On passing, 'Finish' button: Goes to Next Slide

On failing, 'Finish' button: Goes to Next Slide

Allow user to leave quiz: At any time

User may view slides after quiz: At any time

User may attempt quiz: Unlimited times


THANK YOU

module 4 - data domain advanced features and functions_v5

Documents

data domain data retention

data domain vtl

emc confidentialinternal

data domain snapshots

data domain os

data domain system

data domain fastcopy

virtual tape library