© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
Industry Trends and Perspectives:What’s Hot For 2008Greg Schulz, Founder & Sr. Analyst, The StorageIO GroupAuthor, Resilient Storage Networks (Elsevier)
The Green and Virtual Data Center (Auerbach) at Amazon.com
© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
Part I – Hot Storage Issues, Trends and Technologies For Channel Pros
What’s the buzz out there?
© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
What’s the buzz out there?
8 Gb Fibre Channel (8GFC), 10GbE, Agent-less, Authentication, Archiving, Backup, Backup Service Provider (BSP), BC/DR, Benchmarking, Blade Servers, Bulk Storage, CAS, Capacity Planning, Capacity Per Watt, Converged Enhanced Ethernet (CEE), CIFS, Cloud Storage, Clustered Storage, CNA, Compliance, Compression, Converged Networks, D2D2D, Data Management, Data Migration, Deduplication, Dedupe Debates, DPM, e-Discovery, Environmental Health and Safety (EHS), Encryption, Event Correlation, e-Waste, Fibre Channel over Ethernet (FCoE), File Management, FLASH, Green, Grid, HA, I/O Virtualization (IOV), InfiniBand, Infrastructure Resource Management (IRM), IOPS Per Watt, IPM & MAID 2.0, iSCSI, Multi-Protocol Storage, Managed Service Provider (MSP), NAS, NFS, NPVID, Partitions, Power Cooling Floor-space EHS (PCFE), PCI SIG IOV, Performance, pNFS, Removable Hard Disk Drive (RHDD), RAID 6, Replication, RoHS, Replication, CDP, SAN, SAS, SATA, Security, Snapshots, SRM, SSD, Tape, Thin Provision, Tier 0, Unstructured Data, VCB, Virtualization, Vmotion, VMware, VTL, WAAS, WADS, WAFS, WADM, Web 2.0 Storage
© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
Lost tapes make good news copy, however, lost or stolen PDAs, USB devices, laptops and HDDs removed from retired storage systems are a growing threatrisk. Reported along withunreported events areon the rise!
Time
Lo
st D
ata
Eve
nts
GrowingAwareness!
A key question is, can you or yourbusiness afford the risk of data being lost or stolen?
Is more data being lost or stolen today than in the past?Industry Trends – Information Security
Lost tapes are actually on the decline, however…
Encryption Data At Rest and In-Flight, FDE and Drive Trust, Host Software and Storage Encrypt, Key Management, Secure Digital Destruction Authorization, Authentication, Identity Management
© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
ServiceBureau
1950s 1960s 1970s 1980s 1990s 2000s 2010sInfo
UtilitiesxSPs
ManagedService
Providers
SaaS Software or Storage as a ServiceGrid, Cloud,
Web 2.0Outsource & In-source
Cell Phones, PDAs, Pocket PCs
Desktop PCs & Laptops
Midrange & Servers
MainframesLPARs/VMs Open Networking Native Linux
From Proprietary to x86 & Hypervisors & Open
From Desktop to Laptop x86 and VMs
Converged Phones & Computers
DistributeConsolidate
DistributeConsolidate
Distribute
Industry Trends – Compute ContinuumWe are in a consolidation phase (again!)
Client Server
© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
Single Large Server
Four Servers
Same FootprintDifferent Generations over Time
• More processing power per footprint (sq. foot or meter, cabinet U)• Performance density improvements and cost reductions continue• Same floor space occupied to host more compute capabilities• More processing cycles will be needed moving forward• More processing cycles per watt of energy consumed• Power, cooling, floor space and EHS improvements
Eight Servers
24 Servers
128 Servers (Blade Centers)
Industry Trends – Increased DensityRising demand, denser solutions, PCFE and complexity
Source: “The Green and Virtual Data Center” (Auerbach)
© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
Common challenges and issues – old vs. new rules!Industry Trends – Shifting Landscapes
Legacy and Transactional Data
Profile: Data is created, worked with and then goes dormant after some period of time with probability of little to no future access or useExamples: Database, email, transactional, general file serving, project-oriented dataSolution: Ideal candidate for archiving off of primary or online storage to off-line and removable media or MAID-based storage combined with purging or deletion of data no longer needed to meet compliance or other commitments
Time
Act
ivit
y
Data Created Data Goes Dormant
8
Web 2.0 and Online Data
Time
Act
ivit
y
Data Created Continued Access
8
Profile: Data is created, worked with and then may go idle briefly, then accessed, then idle, then active, then idle, then active…Examples: Web, reference and lookup, fixed content, Web 2.0 and social networking, media and entertainment, some email, search, seasonal or event and research-based dataSolution: Online storage with variable performance to meet changing workload demands, bulk and clustered storage, MAID 2.0 and IPM-enabled storage, caching
Source: “The Green and Virtual Data Center” (Auerbach)
© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
Hot Storage Topic – Clustered StorageThe many faces of clustered storage – not just for HPC!
© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
Supply Demand
Limits on generate and transmissionRising energy costs and availabilityAging and expensive infrastructureEmerging eco and ETS regulations
Growing data footprintMore servers and storage
Increase density, reduce costEHS, RoHS, WEEE, E-Waste
Balancing Act
See report “Analysis of EPA report to Congress” at www.storageio.com/xreports.htm
Focus on green house gases (GHG)Carbon credit offsetsEcological sustainment Power avoidanceSave moneyGlobal
Concern with available powerEnergy rebates and certificates
Economic sustainmentEnergy efficiencyCost to be green
Local
Confusion leading to inactivity and missed opportunity
Messaging IssuesThe Green Gap
Industry Trends – Growing Green Gap
© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
Requires real resources: server, network, storage, facilities
FAN, LAN, SAN,WAN, MAN
Local PITSnapshots
Switch
FAN, LAN, SAN, MAN, WAN
Block LUNs(iSCSI & FC)
\\SharedVol001
Vol002File Sharing(NFS & CIFS)
ServersStorage
TapeRotation
Virtual Server Environment
Web
LinuxVM
WindowsVM
File
WindowsVM
SQL
WindowsVM
\\Shared
Vol001
Vol002
Blade Servers
Replicate
NetworkedVolumes &File Shares
Mainframes, Open Systems
Power, Cooling, Floor-space, Environmental (PCFE)
EmailMessagingDatabase, DSS
CAD,EDA,SW Dev
SpreadsheetsPPTs, PDFsBilling, E-TailFile Serving
VolCD
VolCC
\\SharedC1
Video/Audio
Hot Storage Topic – IT Virtualization
© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
Part II – Opportunities to Address Your Customers’ Pressing Storage Issues
New and Emerging Technologies and Solutions
© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
Rel
ativ
e P
rote
ctio
n C
ost
Synchronousdata
mirroring, replication, continuous availability,
nonstop processing.
RTO and RPO near or at zero
Asynchronous remote data mirroring,
replication, snapshot and copy. HA failover cluster. Longer RTO
and/or RPO.
Remote tape or disk-to-disk-to- tape backup, copies and
vaulting. Remote archiving. Extended RTO and RPO.
Weeks MonthsDaysHoursMinutesContinuousRTO and RPO timeline and data lifecycle
RTO = Recovery Time Objective = When data can be usable or available
RPO = Recovery Point Objective = What point data is recovered to, how much data can you afford to potentially lose
Tiered Data ProtectionBalance cost, data loss, availability and applicable threats
EmailMessagingDatabase, DSS
CAD,EDA,SW Dev
SpreadsheetsPPTs, PDFsBilling, E-TailFile Serving
VolCD
VolCC
\\SharedC1
Video/Audio
© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
Available PCFE Threshold
Cooling/DistributionServers Storage
NetworkingOther
PCFE Capacity Threshold CeilingAvailable IT Resources/Demand
PCFE Constrained
IT Capacity ConstrainedBusiness Growth Inhibited
Economic PenaltiesLost Opportunity
Time
Compute Capacity Storage CapacityI/O Performance (IOPS & Bandwidth)
Relative Available IT Resources
Industry Trends – PCFE IssuesRising demand, denser solutions, more PCFE supply issues
Source: “The Green and Virtual Data Center” (Auerbach)
© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
PCFE = Power, Cooling, Floor space, EHSHot Storage Topic – PCFE aka “Green”
Wheel of Opportunity: Economic and Ecological Sustainment
Consolidation
MaskOr MoveIssues
Reduce Data Footprint
Boost Energy Efficiency
EnergyAvoidance
TieredServersStorage
HVACAlternative
Energy
E-WasteEHS
RoHS
Fin
an
cia
l In
cen
tiv
es a
nd
Re
bat
es
Be
st
Pra
cti
ces
an
d P
olic
ies
/ M
etr
ics
Source: “The Green and Virtual Data Center” (Auerbach)EHS = Environmental Health Safety
© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
Infrastructure Resource Management(IRM) Functions and Activities
• Namespace and virtualization• Measurements and metrics• Monitoring and reporting• Modeling, analysis, planning• Resource usage and allocation• Performance and capacity planning• Thin provisioning and purposing• Diagnostic and resolution• Change and configuration validation• Data protection and footprint reduction• Policy management and service levels• Facilities and asset management• Logical and physical security• Procurement and disposition
IT Infrastructure Resource Management (IRM)
Processes and ToolsProcesses and Tools
Hot Storage Topic – Cross Domain IRM
Source: “The Green and Virtual Data Center” (Auerbach)
© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
Usage CasesHow Measured
Best Practices
Categories &Price Bands
Easy to UseReflective of
Diversity of StorageActive & Idle, etc.
Needs toBe Applicable
to UsageModel
Metrics and MeasurementsWhy, where, how and what to measure now and future
Source: “The Green and Virtual Data Center” (Auerbach)
© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
Processor to disk storage capacity
Disk storage capacity to I/O performance gap
Time
Pe
rfo
rma
nc
e
Server processor performance curveDisk storage capacity curveDisk storage performance curve (IOPS)
Growing server processing to storage capacity and I/O gapIndustry Trends – I/O Performance Gap
See “Addressing Data Center I/O Performance Gap” www.storageio.com/xreports.htm
© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
FootprintPACECost
Service Level
Balancing Act
PCFE
Tiered Storage and Tiered AccessBalance performance, availability, capacity, energy usage
RAMFLASH
Fast HDD
FAT HDD
Tape &Optical
Tier-3Tier-0 Tier-1 Tier-2
Power
Pric
e B
ands
EnterpriseMidrangeSMBSOHO
Performance
Price
Different Price Bands and Categories
Rel
ativ
e C
ompa
rison
Si
mila
r Cap
aciti
es
ConsolidateSpace Capacity
Accelerate Performance“Time is Money”
Reduce Capacity
Costs
EmailMessagingDatabase, DSS
CAD,EDA,SW Dev
SpreadsheetsPPTs, PDFsBilling, E-TailFile Serving
VolCD
VolCC
\\SharedC1
Video/Audio
Source: “The Green and Virtual Data Center” (Auerbach)
© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
I/O virtualization – converged networks, IOV and FCoE
Traditional ApproachesSeparate Networks & Interconnects
(Fibre Channel, GbE, IBA, Etc.)
Evolving ApproachesUnified & Converged Interconnects
(Virtualized FC, GbE, FCoE, Etc.Physical Data Center Ethernet or IBA)
Hot Storage Topic – I/O Virtualization
I/O, I/O, it’s off to virtual work we go…
© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
• Sharing of physical adapters and unique addressing– Fibre Channel N_Port Virtual IDs – Unique N_Port addresses– Virtual Machine NICs and HBAs – E.g. VMware, Virtual Iron
• Virtual NICs and Virtual HBAs for virtual machines
• PCI bus extension, switching and sharing – very short distances– PCI SIG SR-IOV and MR-IOV: Share PCI adapters
• Converged I/O and networking– Virtual and converged adapters and NICs
• Transform physical adapter into virtual adapters and NICs– Converged networks and fabrics
• InfiniBand and Converged Enhanced Ethernet and FCoE
Fibre Channel over Ethernet (FCoE); Single-Root IOV (SR-IOV); Multi-Root IOV (MR-IOV)
Many Faces of I/O VirtualizationI/O Virtualization and Consolidated I/O and Networking
© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
Sparse, Duplicate Files and Content
Original Data
10 TB
Primary Database, Email, File serving
8 TB 2 TB
RAID 1+0 RAID 1 RAID 5
App-a App-b App-c
More data being stored, copied, backed up, protectedIndustry Trends – Data footprint growth
See “Business Benefits of Data Footprint Reduction” www.storageio.com
Copies – Data proliferation and expanding data footprintBackup, BC, DR, HA, Archive, Compliance
DSS, Training, Test, Dev QA, Operational, Needs
10 TB10 TB 8 TB
10 TB 8 TB 2 TB
10 TB 8 TB 2 TB10 TB 8 TB 2 TB
10 TB 8 TB 2 TB10 TB 8 TB 2 TB
10 TB 8 TB 2 TB10 TB 8 TB 2 TB
10 TB 8 TB 2 TB
10 TB 8 TB 2 TB10 TB 8 TB 2 TB
In addition to storage space capacity, IOPS and MBPS to move data needs to be considered.
Challenge: More data to back up, protect and manageSolution: Reduce footprint impact: Archive, compress, dedupe, tiered storage
Multiple data footprint by data protection factor (e.g. RAID level and mirroring) along with spares and free space to account for actual disk space
Source: “The Green and Virtual Data Center” (Auerbach)
© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
• Develop an overall data footprint reduction strategy– Address online primary, secondary and off-line data– Combine archiving, compression and dedupe in strategy
• Archiving and pruning with data classification– Database, email, unstructured block and file– Big benefit, costly to deploy (people, hardware, software)
• Compression for online and off-line storage– Can be applied to most any data type for some benefit– Online real-time compression for active primary data
• Single Instance Storage (SIS) and deduplication– Initially targeted at backup and archive– Look into scaling capabilities and claims by vendors
See “Business Benefits of Data Footprint Reduction” www.storageio.com
Hot Storage Topic – Data Footprint ReductionArchiving, compress (on- and off-line), deduplication
© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
Market and IT Opportunity = Life Beyond ConsolidationUsing server virtualization for IT resource management
Enabling abstraction and transparency for massive scaling, BC, DR and routine infrastructure resource management (IRM)
operational functions
Total Serverand StorageMarket Size
Market and ITVirtualization Opportunity!
Non-ConsolidatedServers or Storage
• Emulation• Abstraction• Aggregation• Migration
Tomorrow
Only a fraction of allservers or storageconsolidated!
Issues that inhibit consolidation:• QoS and performance barriers • Politics and financial constraints • Competitive or legal purposes• Security and compliance
Consolidated Server and Storage Using
Virtualization
Today
The Many Faces of VirtualizationConsolidate, Abstract, Emulate, Migrate and Manage
Source: “The Green and Virtual Data Center” (Auerbach)
© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
Consolidation with HARedundant Servers
ConsolidationSingle Server
Scaling Beyond a ServerSoftware Changes
Scaling Beyond a ServerSingle Operating System
The Many Faces of VirtualizationConsolidate, Abstract, Emulate, Migrate and Manage
Source: “The Green and Virtual Data Center” (Auerbach)
© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
The Many Faces of Server VirtualizationLeveraging Virtualization to Enable HA, BC and DR
Traditional BC/DR1 to 1 Resource Allocation
Or Selective Recovery
PM = Physical Machine
Network
Virtualized BC/DRInitially Oversubscribed Add
Physical Resources As Needed
VMs VMs VMsPMs PMs PMs
VMs VMs VMsPM
ProductionPrimary Site
32 Physical Servers32 VMs 1:1 VM to PM
BC/DRSite
8 Physical Servers32 VMs 4:1 VM to PM
LocalSharedStorage
Snapshots and
ReplicationRemoteSharedStorage
Network
PMs PMs PMs PMs PMs PMs
ProductionPrimary Site
32 Physical Servers
BC/DRSite
32 Physical Servers
LocalSharedStorage
Snapshots and
ReplicationRemoteSharedStorage
Data Protection Management
Source: “The Green and Virtual Data Center” (Auerbach)
© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
The Virtual Data CenterDifferent approaches and locations for virtualization
Remote backupand archives
InternetMAN & WAN
RemoteStorage
Clustered andnon-clustered
servers
Data replicated for HA, BC, DR
Firewalls
• Data movement and access between sites• Remote-office/branch-office data access• HA, BC, DR for business sustainment• Leverage off-site managed services• Remote data archiving
Managed Service Provider (MSP), Software or Storage as a Service (SaaS)
Wide Area, Internet and Cloud Networking
• Privately owned facilities• Hosted or colocation• BC/DR standby hot/cold site• Cloud or SaaS and MSP
Source: “The Green and Virtual Data Center” (Auerbach)
© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
What You Can Do Today and TomorrowBoost energy efficiency – more work per energy used
Source: “The Green and Virtual Data Center” (Auerbach)
© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
What you can do now to address customer issues
Part III – Call to Action
How to stand out in the crowd!
© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
• Gain control of virtual and physical resources– More data to protect for longer periods of time– Fill in the gaps and add plug-ins to virtualization frameworks– Leverage new technologies and capabilities
• Tiered servers, storage, networks, data protection and access• Data protection management including site recovery manager• High-availability virtual server and storage migration• Data footprint reduction: Archive, real-time compress, dedupe• Disk-based snapshots, backup, replication and archiving• Disk-based virtual tape libraries and removable media• Encryption of fixed and removable media• Virtualization and application aware data protection
What You Can Do Today!Virtual environments need physical resources and IRM
© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
Different Facets of Server VirtualizationConsolidation or Aggregation, Emulation and Transparency
Migration
Virtual Infrastructure
WebAppLinuxVM
EmailWindows
VM
FileWindows
VM
AppsWindows
VM
Disk StorageDAS, SAN or NAS
Servers orBlade Centers Migration
Virtual Infrastructure
Consolidation of Multiple SystemsBoost Utilization of Servers or Storage
HA, BC, DRLoad Balancing
EmailMessagingDatabase, DSS
CAD,EDA,SW Dev
SpreadsheetsPPTs, PDFsBilling, E-TailFile Serving
VolCD
VolCC
\\SharedC1
Video/Audio
FileWindows
VM
AppsWindows
VM
Replicate
Enable: Physical to Virtual (P2V), Virtual to Virtual (V2V)
Requires: 3rd party Replication, Backup, Snapshots, Data Protection Management, Shared Storage
Source: “The Green and Virtual Data Center” (Auerbach)
© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
Vol-DVol-C
Migration
Virtual Infrastructure
WebAppLinuxVM
EmailWindows
VM
FileLinuxVM
AppsUNIXVMPre/Post
Processing
VTL / Disk Library(Block or File) Tape Devices
Disk Storage
BackupServer
LAN, MAN or WAN
Vol-BVol-ASnapshots
Data MovementOver SAN or DAS
Hot Storage Topic – Data ProtectionOpportunity to re-architect data protection practices
VCBs, Snapshots, Replication, Application
Aware, HA vs. BC/DR
D2D, D2D2D, D2T, D2D2T, VTLs, Encrypt, Compress & Dedupe
Data ProtectionManagement ToolsManaged Services
© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
Mask or Move Issues Outsource, MSP, Carbon CreditsConsolidation Virtualization and Aggregation, IOVReduce Data Footprint Archive, Compression, Dedupe
Tiered Storage RAID Levels, SSD, HDDs, Optical, TapeEnergy Avoidance Power Down, MAID, IPMBoost Energy Efficiency More Performance, Less Power, AVSHVAC, Alternate Power Hot/Cold Aisles, CRACE-Waste and Hazmat RoHS, Recycling, WEEE, EHSFinancial Incentives Rebates and Efficiency IncentivesMetrics/Measurements Insight into Energy EffectivenessBest Practices/Policies Improve Usage of IT Resources
AVS = Adaptive Voltage Scaling IPM = Intelligent Power Management
What You Can Do TodayAddressing different issues – no single magic bullet!
© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
• Look beyond consolidation to enable and sustain growth– Be in position to support future growth. It’s just a matter of time!
• You can’t go forward if you can’t go back!– Time to re-architect data protection, BC and DR with new techniques.
• You can’t delete what you have not preserved– Assuming customer data has some value, preserve before deletion.
• Archiving is for more than regulatory compliance purposes– Implement tiered storage, boost performance, address PCFE.
• You can’t manage what you or your customer don’t know about– Identify what data, files and objects you have: insight.
General CommentsBasic premises – gain management insight and control
Lean more at www.storageio.com and www.thegreenandvirtualdatacenter.com
© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
• Many approaches depending on your customers’ issues– Help customers do more with what they have to sustain growth– Help customers shift from energy avoidance to energy efficiency
• Avoid simply moving IT problems around– Help your customers gain control of data and infrastructure issues– Solve problems and issues to enable your customers to grow
• Instead of race to replace tape, revamp data protection architecture• Instead of race to consolidate data centers, enable remote management
• Balance between future and what works today– Leverage what works and what customers are buying– Land sales and revenue on shipping products while selling the future– Help your customers develop strategies for moving forward
Closing CommentsBasic premises – gain management insight and control
© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
•Where you can learn more:– Feel free to call or email me with questions or comments– See Resilient Storage Networks (Elsevier)– www.storageio.com (see portfolio for articles etc.)– www.thegreenandvirtualdatacenter.com
Closing CommentsData protection, PCFE/green topics are here to stay!
“The Green and Virtual Data Center” (Auerbach)Order your advance copy now on Amazon.com
© Copyright 2008 StorageIO Group All rights reserved. www.storageio.com
Thank You!
www.storageio.com