enterprise vtl strategies & data duplication for business ... · - does the vtl tunes itself to...
TRANSCRIPT
NCS Corporate Presentation - NCS CONFIDENTIAL Copyright©1996-2007 NCS Pte Ltd. All Rights Reserved.
Enterprise VTL Strategies & Data Duplication for Business ContinuityEnterprise VTL Strategies & Data Duplication for Business Continuity
:: Welcome to NCS ::
NCS Corporate Presentation - NCS CONFIDENTIAL Copyright©1996-2007 NCS Pte Ltd. All Rights Reserved.
Presented byBernard Lee
Consultant (NCS Enterprise Storage Group)
31 Jan 2008
2
NCS Corporate Presentation - NCS CONFIDENTIALCopyright©1996-2008 NCS Pte Ltd. All Rights Reserved.
Explosive Network & Data Growth
30 Billion packages shipped per year2 Billion text messages on a peak day
390 Gigabytes of data created every second50% New data growth each year
3 Million People a Week Join the Network !3 Million People a Week Join the Network !
4
NCS Corporate Presentation - NCS CONFIDENTIALCopyright©1996-2008 NCS Pte Ltd. All Rights Reserved.
Changing Customer Expectations
�Explosion of persistent data within company networks
�Escalating power and cooling costs
�Tape no longer meets customer needs for cost-effective, highly reliable, long-term storage
�New regulatory and compliance demands
�Data Protection / BC / DR
5
NCS Corporate Presentation - NCS CONFIDENTIALCopyright©1996-2008 NCS Pte Ltd. All Rights Reserved.
� Cost
– Backup/Restore costs increased year on year
� Low Performance
– Explosive growth of data
– Over 80% of restore requests within 48 hours
– Back-up windows shrinking
� Complexity/Management
– 70% of storage mgmt is Backup/Archive
– 15% of storage admin's time is spent on recovery
� Security/Trust
– 5-20% of Backup/Restore jobs fail nightly
Customers' pains around data protection are growing
6
NCS Corporate Presentation - NCS CONFIDENTIALCopyright©1996-2008 NCS Pte Ltd. All Rights Reserved.
Reduce the complexity of managing today's complex data and storage environment
Store, Protect, Secure, Access, Archive, Comply and Shred
Complexity = Cost & Risk
IT Challenges
7
NCS Corporate Presentation - NCS CONFIDENTIALCopyright©1996-2008 NCS Pte Ltd. All Rights Reserved.
Traditional Backup and Tape-Library Challenges
� Performance of backup and restore
– Not meeting backup windows
– Cannot provide adequate restore service levels
� Reliability and availability of tape media
– Libraries, drives, and cartridges fail
– Tape does not support RAID protection
� Management complexity
– Constant tuning of environment
� Risk associated with media management
– Lost tapes
– Time to find media for restore
– Cost of transporting tapes
8
NCS Corporate Presentation - NCS CONFIDENTIALCopyright©1996-2008 NCS Pte Ltd. All Rights Reserved.
Traditional TradeOff – Tape Vs Disk
Costvs
SLAs
Traditional Tradeoff - Tape vs. Disk
� Cheap� Easy� Inertia� Well understood� It’s the way we’ve always done it� Good enough� Portable
� Expensive� More than we need� Not Portable� Difficult to manage� Not how we do it
9
NCS Corporate Presentation - NCS CONFIDENTIALCopyright©1996-2008 NCS Pte Ltd. All Rights Reserved.
Tipping Point to move to disk
Tipping Point to Move to Disk
Costvs
SLAs
� Compliance� Lower-cost bandwidth� Low-cost hard drives� Need a better solution
capable of improving on organizational efficiency
� Unreliable� Not secure� Slow� No leverage of
information beyond storage
10
NCS Corporate Presentation - NCS CONFIDENTIALCopyright©1996-2008 NCS Pte Ltd. All Rights Reserved.
Traditional Backup, Recover and Archiving Process – Offsite Tape Storage
Tape Storage Site Production Data Center
RecoverySite
CTAM
• Recovery is complex• Recovery takes a long time• Risk of data loss• Risk of security breach
CTAM
11
NCS Corporate Presentation - NCS CONFIDENTIALCopyright©1996-2008 NCS Pte Ltd. All Rights Reserved.
Where are VTLs needed
� Data Centers with
- Constrained backup windows
- Stringent Response Time Objectives (RTOs)
- High Growth rates
• Add capacity dynamically
• Defer purchase on additional tape drives and slots
� Remote Offices
- Eliminate manual tape handling
- Consolidate backups to main data center
12
NCS Corporate Presentation - NCS CONFIDENTIALCopyright©1996-2008 NCS Pte Ltd. All Rights Reserved.
How Virtual Tape Libraries Work
� VTLs are disk systems that emulate tape drives and l ibraries
- Backup application writes to VTL as if it were tape drives
- Requires few, if any, changes, to backup application
� Solves traditional backup problems
- Tape devices run optimally when streaming
• VTLs improve backup performance of small files
• Improves restore time for small files
13
NCS Corporate Presentation - NCS CONFIDENTIALCopyright©1996-2008 NCS Pte Ltd. All Rights Reserved.
� Lack of Scalability
- only way to scale is by adding controllers, coupling of performance and
capacity is costly
� Complex Installation and Configuration
� Lack of Data Compression for Optimized Backup-Up performance
� No On-Disk Data De-duplication
Limitations of 1 st Generation VTLs
14
NCS Corporate Presentation - NCS CONFIDENTIALCopyright©1996-2008 NCS Pte Ltd. All Rights Reserved.
� Performance
- Maximum Throughput of disk-based backup system
- Performance benchmark
- How are backup images written to the backend disks
- Does the VTL tunes itself to deliver optimal performance
- How many disk drives are required to achieve maximum performance
� Integration with Physical Tape
� Compression
� Scalability
� Replication
� Tape Drive and Library emulation
� Management software
� Market acceptance
� Vendor Support
� Investment Protection
Evaluating Virtual Tape Libraries
15
NCS Corporate Presentation - NCS CONFIDENTIALCopyright©1996-2008 NCS Pte Ltd. All Rights Reserved.
VTL Disk Capacity Sizing
Assumptions:� Primary storage of 5TB to be backed up fully every weekend� Five daily incremental backups per week – 100GB each� 4 weeks of disk-based onsite capacity� Raid5 Disk storage in a 4+1 configuration� 2:1 compression of backup data from the backup application
Disk-based Backup Capacity:4 weeks * ( 5TB + (5*100GB) ) = 22TBRAID5 overhead + 20%, Total = 26.4TB2:1 Compression, Total = 13.2TB
16
NCS Corporate Presentation - NCS CONFIDENTIALCopyright©1996-2008 NCS Pte Ltd. All Rights Reserved.
EMC Disk Library Family
Scalability
Ser
vice
Lev
els
DL210 � Single Disk Library engines� 4 x 2 Gb/s Fibre Channel ports� 2 Gigabit Ethernet ports� Up to 72 drives� Up to 24 TB (usable)� Up to 72 TB (compressed)� Up to 1.368 TB/hour
� 1–2 Disk Library engines� 8–16 4 Gb/s Fibre Channel ports� 6–12 Gigabit Ethernet ports� Up to 930 drives� Up to 674 TB (usable)� Up to 1.8 PB (compressed)� Up to 7.92 TB/hour� RAID 6 or RAID 5 models
DL4000 Series
� 2–4 Disk Library engines� 16–32 4 Gb/s Fibre Channel ports� 12–24 Gigabit Ethernet ports� Up to 2,400 drives� Up to 615 TB (usable)� Up to 1.8 PB (compressed)� Up to 11.4 TB/hour
DL6000 SeriesThe Easiest Way to Improve Backup and Restore
17
NCS Corporate Presentation - NCS CONFIDENTIALCopyright©1996-2008 NCS Pte Ltd. All Rights Reserved.
EMC Disk Library Family
Consolidated media management
Replication
Environmental support—more than three million configurations
Active Engine Failover
11.4 TB/hr6.4 TB/hr7.3 TB/hr3.7 TB/hr3.7 TB/hr1.3 TB/hrPerformance (uncompressed)
584 TB615 TB674 TB/520 TB
337 TB/260 TB
337 TB/260 TB
24 TBUp to capacity (usable)
Symmetrix DMX
Symmetrix DMX
CLARiiON CX3-80
CLARiiON CX3-80
CLARiiON CX3-80
CLARiiON AX150
Storage
DL6300DL6100DL4406/DL4400
DL4206/DL4200
DL4106/DL4100
DL210
New New New
18
NCS Corporate Presentation - NCS CONFIDENTIALCopyright©1996-2008 NCS Pte Ltd. All Rights Reserved.
Capacity on Demand
� Ability to allocate disk storage to virtual tapes upon write, not creation
� Disk library storage is not consumed by empty virtual tapes
� Virtual tapes are filled by fixed increments (5 GB)
� Virtual tape data volume matches used storage
• Improves storage utilization within EMC Disk Library
• Lowers cost by not allocating space to blank tapes
Storage consumed upon write
19
NCS Corporate Presentation - NCS CONFIDENTIALCopyright©1996-2008 NCS Pte Ltd. All Rights Reserved.
Broadest Environmental Support in the Industry� Tape Libraries
– ADIC Scalar24 and 100, i500, i2000 Series
– Sony PetaSite
– ATL P-Series, ATL 7100
– STK L-Series, SL500, STK 97xx, ACSLS
– IBM 3590, 3584, 3583
– Overland NEO2000, NEO4100
– Quantum M1500, M1800
� Tape Drives
– DLT, SuperDLT, AIT, SAIT
– LTO, LTO2, LTO3, LTO4 IBM 3590, IBM 3592, IBM S1120
– Sun StorageTek 9840, 9940, T10000
� Backup Applications
– EMC NetWorker, EMC Data Manager
– Symantec Veritas NetBackup and Backup Exec
– IBM Tivoli Storage Manager
– BRMS, OS/400 Native
– CommVault Galaxy
– CA ARCserve, BrightStor Enterprise Backup
– Plus many more: SyncSort, BakBone, Atempo, HP…
20
NCS Corporate Presentation - NCS CONFIDENTIALCopyright©1996-2008 NCS Pte Ltd. All Rights Reserved.
Real World Deployments
Customer A
21
NCS Corporate Presentation - NCS CONFIDENTIALCopyright©1996-2008 NCS Pte Ltd. All Rights Reserved.
Real World Deployments
Fabric 1Fabric 1 Fabric 2Fabric 2
Customer B
22
NCS Corporate Presentation - NCS CONFIDENTIALCopyright©1996-2008 NCS Pte Ltd. All Rights Reserved.
Reason For Growing File System Data
• Keeping inactive data on primary storage is increasingly costly and inefficient
• Primary storage is typically more expensive
• Backups and restores take longer and contribute to storage consumption
• Application performance can suffer
Inactive Data
Active Data
51% of open system data is unnecessary, duplicate, or
non-business related
70% of data on Windows has not been accessed for 90 days or more
Source: SNIA/Source Consulting
60% improvement in management efficiency is needed every year to keep up with storage growth
55% of unplanned server outagesoccur from disk space consumption
23
NCS Corporate Presentation - NCS CONFIDENTIALCopyright©1996-2008 NCS Pte Ltd. All Rights Reserved.
De-Duplication� De-duplication
Called by many names- Compaction, data de-duplication, single instance store, coalescence, de-dupe
� Different implementations
� Process- Examine incoming stream- Removes repetitive files and replaces removed files with pointers to original file- Eliminates redundant attachments in emails- Eliminates redundant unchanged files
Types of de-duplication
� Source de-duplication
� Target de-duplication
24
NCS Corporate Presentation - NCS CONFIDENTIALCopyright©1996-2008 NCS Pte Ltd. All Rights Reserved.
Types of De-duplication
Requires hardware at each remote site to be protected via de-dupe
� Fast implementations� Does not require change in backup software
Target De-duplication
� Required change of backup software� Typically slower than target de-dupe on large volumes of data (TB)
� Reduces bandwidth usage all over� Can protect remote office without any hardware installed there
Source De-duplication
DisadvantageAdvantage
25
NCS Corporate Presentation - NCS CONFIDENTIALCopyright©1996-2008 NCS Pte Ltd. All Rights Reserved.
De-Duplication
DeDe--duplication can occur at many levelsduplication can occur at many levels
1.1. At the protected computerAt the protected computer
2.2. At the backup server/repositoryAt the backup server/repository
3.3. In a disk device or VTLIn a disk device or VTL
26
NCS Corporate Presentation - NCS CONFIDENTIALCopyright©1996-2008 NCS Pte Ltd. All Rights Reserved.
Key Criteria for an Enterprise De-Duplication Solut ion
The de-duplication algorithm should be able to hand le any data that is sent to it, irrespective of source and cont ent
Source-Agnostic
The solution must fit with existing practices, poli cies and SLAsNon-Disruption
Risk of data corruption that is associated with the de-duplication process must be zero
Data Integrity
Solutions must allow for large and growing (hundred s of TB) repositories.
Capacity
Solutions must allow customers to process more data fasterPerformance
Technology should efficiently find and eliminate re dundant dataDe-DuplicationEfficacy
DescriptionKey Criteria
27
NCS Corporate Presentation - NCS CONFIDENTIALCopyright©1996-2008 NCS Pte Ltd. All Rights Reserved.
Avamar Product Description
� Avamar software revolutionizes the backup process w ith global data de-duplication technology
� Reduces the size of the backup data at the source—b efore it is transferred across the network—cutting backup time by up to 10 times
� De-duplicates across sites and servers for maximum efficiency
� Customers can reduce required bandwidth—and the amo unt of data backed up daily—by up to 500 times
� Highly effective de-duplication allows customers to cost-effectively store a greater number of full backups on disk—reducing or eliminating the need for tape
� Ideal for the protection of remote offices, virtual environments, and bottlenecks in the data center—enabling fast, s ecure backups over existing network connections
28
NCS Corporate Presentation - NCS CONFIDENTIALCopyright©1996-2008 NCS Pte Ltd. All Rights Reserved.
How it WorksGlobal Data De-Duplication
� Break data into atom (sub-file, variable-length segments of data)
O
H H
O
H H
O
H H
O
H H
O
H H
O H
� Send and store each atom only once
� Avamar backup repository
At the source —De-duplication before data is transported across t he networkAt the target —Assures coordinated de-duplication across sites, s ervers, and over timeGranular —Small, variable-length segments guarantee most eff ective de-duplication
29
NCS Corporate Presentation - NCS CONFIDENTIALCopyright©1996-2008 NCS Pte Ltd. All Rights Reserved.
Deployment Example
WAN
(ENCRYPTED)
(ENCRYPTED)
Data Center
Primary systems
A
Avamar Data Store
(ENCRYPTED)
Remote Recovery Site
Primary systems
A
TapeVault
Avamar Data Store
Small Remote Site
Avamar agent onlyon primary systems
APrimary systems
A
Avamar Data Store
(ENCRYPTED)
Avamar Software AgentA
30
NCS Corporate Presentation - NCS CONFIDENTIALCopyright©1996-2008 NCS Pte Ltd. All Rights Reserved.
Supported Client Platforms and Applications
10.4.xMac OS X (PowerPC and Intel)
8.2IBM UDB DB2
DART 5.5Data ONTAP 6.3, 6.4, 6.5, 7.0.4, 7.0.5, 7.0.6
NDMP for EMC Celerra and NetApp Filers
9i, 10g, 10gR2Oracle
7.0, 2000Microsoft SQL Server
5.5, 2000, 2003Microsoft Exchange
8, 9SUSE Linux Enterprise Server
8.0, 9.0Red Hat Linux
2.1, 3.0, 4.0Red Hat Enterprise Linux
11.0, 11iV2, 11iV2HP-UX
5.1, 5.2, 5.3AIX
2.6, 7, 8, 9, 10Sun Solaris
XP, NT 4.0, 2000, 2003Microsoft Windows
All versions for supported guests; VMware ESX Server 2.5.3, 3.0.1; VMware Consolidated Backup
VMware
VersionsPlatform
31
NCS Corporate Presentation - NCS CONFIDENTIALCopyright©1996-2008 NCS Pte Ltd. All Rights Reserved.
Avamar Data Store
Prepackaged, Preconfigured Deployment Option for Av amar
� Complete EMC backup and recovery solution
– Avamar de-duplication software
– EMC-certified hardware, fully configured and delivered
� Built-in high availability
– Avamar RAIN technology
– Spare node, RAID
– Redundant power
� Simplifies
– Purchase (single vendor, certified hardware)
– Deployment (minimizes onsite setup)
– Service (single vendor support)
32
NCS Corporate Presentation - NCS CONFIDENTIALCopyright©1996-2008 NCS Pte Ltd. All Rights Reserved.
Flexible Deployment Options
� Avamar Data Store
– Individual Avamar Data Store scales to equivalent of up
to 486 TB of traditional backup storage*
– Backup media requirement reduced 20–30 times
– Each two-node expansion adds the equivalent of 70 TB of traditional backup storage*
� Avamar Data Store, single node
– Supports equivalent of up to 37 TB of traditional backup storage*
– Designed for easy deployment at remote offices
– Offers fast, local recovery without dependence on a WAN connection
Avamar Data Store
33
NCS Corporate Presentation - NCS CONFIDENTIALCopyright©1996-2008 NCS Pte Ltd. All Rights Reserved.
Tangible Customer Results
Stop the Avalanche Before It Starts: Benefits Begin at the Source
7,831 GB
9,583 GB
3,265 GB
5,097 GB
3,573 GB
Amount of Primary Data
104.2 GB
80.0 GB
24.2 GB
11.7 GB
6.1 GB
Amount Moved Daily
75:1
120:1
135:1
436:1
586:1
DailyDe-dupe Ratio
Mix of Linux file systems and databases
Mix of 20% databases, 80% file systems (Windows and UNIX)
Engineering files on NAS Filers (NDMP backups)
Mix of Windows, Linux, and UNIX file systems
Windows file systems
Data Type
Source: EMC
34
NCS Corporate Presentation - NCS CONFIDENTIALCopyright©1996-2008 NCS Pte Ltd. All Rights Reserved.
Avamar Transforms Backup and Recovery
� Global data de-duplication defuses the explosion of backup data
– Up to 500:1 daily reduction in network
impact
– Radically reduce time and storage
required for backup
� Alter the fundamental economics of disk versus tape
– Accelerate shift to disk as primary
medium for backup
� Alternative to archaic IT processes(shipping tapes for disaster recovery)
– Automated, encrypted remote copy over
existing WANs