beyond nas and san: the evolution of storage marc farley author building storage networks
TRANSCRIPT
![Page 1: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/1.jpg)
Beyond NAS and SAN:The Evolution of Storage
Marc FarleyAuthorBuilding Storage Networks
![Page 2: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/2.jpg)
What We’ll Do
Analyze storage network functions
Apply these functions in NAS and SANs
Observe market pressures forcing
changes
Identify evolving architectures
![Page 3: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/3.jpg)
Fundamental Functions of Storage Networks
Wiring• Fibre Channel. Parallel SCSI, Ethernet, IP
Storing• Devices & Subsystems, I/O Controllers,
Virtualization SW & HW (RAID, Volume Managers,
Mirroring)
Filing• File systems, Databases, Backup, Replication
Building Blocks
![Page 4: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/4.jpg)
WiringStoringStoring
FilingFiling
Traditional SAN Functional Distribution
Host SystemStorage
SubsystemFibre Channel
Network
HBA
Building Blocks
Controller
![Page 5: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/5.jpg)
WiringStoringStoring
Traditional NAS Functional Distribution
Client System File ServerSystem
LAN
FilingFilingFilingFilingFilingFiling
NICInternal Bus
Building Blocks
![Page 6: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/6.jpg)
Wiring: Physical & Logical Parts
Physical = Cabling and Network Hardware• signaling, encoding, error detection
Logical = Algorithms, Addressing,
Protocols and Services• access methods, flow control (buffer management)
• addressing, naming, topology management, routing
• filtering, zoning
• gateway, bridging and tunneling services
Building Blocks
![Page 7: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/7.jpg)
Wiring: Fibre Channel and i-SCSI
Functionally equivalent as wiring technologies
More of a business issue than a technology
issue
Question: Will i-SCSI SANs be implemented as
distinct networks or grow as part of the
existing Ethernet/IP infrastructure?
i-SCSI leverage might be less than expected
- but it might be good enough
Building Blocks
![Page 8: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/8.jpg)
Wiring: Separate SANs or Bigger LANs?
LAN
LAN SANGood Idea
![Page 9: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/9.jpg)
An Question for the Audience…..
What is the best routing method for
storage traffic?• To date, storage networks have borrowed from IP
networking
• OSPF in IP networks (NAS) & FSPF in Fibre Channel
• What do you want?
• Fast recovery & optimal availability
– OR
• Compatibility with IP routing
An Idea
![Page 10: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/10.jpg)
Storing
Block level operations• storage block addressing
• store, retrieve, status, resource reservation
Subsystems and Virtual Devices• block translation, mirroring, multiple ports, LUN masking
I/O commands between initiators and targets• SCSI, serial SCSI, SCSI adaptations, IDE
• Error correction reallyreally happens here
Building Blocks
![Page 11: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/11.jpg)
Storing on Steroids: Virtualization
Virtualization techniques expand and
extend the capabilities of devices and
subsystems
Virtualization as a storing level function
has no ability to manage anything about
its contents
VirtualStorage
Physical Devices
![Page 12: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/12.jpg)
Filing
Representation of data• files, directories, tables, records
Storage data structure (block address
layout) • equalized performance, maintain free blocks
System kernel integration• kernel manages time, file system manages space
File semantics• Open, close, update, delete, block range operations
Building Blocks
![Page 13: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/13.jpg)
The Fundamental Role of Filing
Manage the use of the storing address
space
Building Blocks
StoringStoringFilingFiling
![Page 14: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/14.jpg)
Internal Functions in Filing
Data View
MetadataData
Structure
![Page 15: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/15.jpg)
NAS & SAN REDUX
![Page 16: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/16.jpg)
SANs are the Application of Storing Functions Over a Network
The Caveat: SANS do not convey any
knowledge of data structures • & there is no way to provide data-level managementGotcha
StoringStoring Functions
Initiators Targets
Storing
Network
![Page 17: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/17.jpg)
Virtualization Creates Storing Structures
Block Data
Initiators Virtual Targets
Storing
Network
Storing
Physical Devices
But not data
structures
![Page 18: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/18.jpg)
NAS: The Application of Filing Functions over a Network
The Caveat: ‘Loose wiring’ for
I/O intensive computing
Gotcha
Filing Functions
Clients Servers
FilingFiling FilingFiling
Network
![Page 19: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/19.jpg)
Challenge for NAS: Distribute Filing Functions & Reinforce the Wiring
File Data
Clients ServersNetwork
![Page 20: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/20.jpg)
Storage Network Requirements
![Page 21: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/21.jpg)
Market Needs
Integrity
Recoverability
Security
Availability
Manageability (Scalability)
Best Practice
![Page 22: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/22.jpg)
Data Integrity Must be Maintained
Error free transmissions• Fiber optic cabling
Data segregation is a good thing• Who wants a Battle Royale over data?
• Any to any connectivity is not required
• Storage networks differ a lot from data networks
• Zoning, masking and fencing are band-aids
for existing architectural weaknesses
• Filing enforces data integrity through locking
Gotcha
![Page 23: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/23.jpg)
Data Recoverability = Redundancy
Data redundancy system redundancy
Mirroring is fast, but relatively inefficient • Extended distance mirrors are “open territory”
• System buffer management is a caveat
Data structures must have integrity• Journalled file systems, database commitments
File system intelligence is most efficient• Serverless backup, data snapshots, replication
Gotcha
![Page 24: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/24.jpg)
Security - the Great Afterthought
Protecting Data From Theft
File systems have done this for decades• System-login is the gatekeeper
There are no constructs for storing-level
security
Wiring-level security can be done, but is slow
Encryption is S-L-O-W for I/O• IP-SEC may provide a method
Gotcha
![Page 25: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/25.jpg)
Remember the Good Old Days of SCSI?
Data integrity was less exposed with
Parallel SCSI • Human error was far less likely
• Storage was not accessible to hackers
Processing power on the storage end is
needed to provide security functions
![Page 26: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/26.jpg)
Data Availability
Remove the system as a bottleneck• (And integrity and security become more suspect)
Availability doesn’t necessarily mean
“seamless”• Single servers in SANs can failover using redundant paths
• NAS network failures must re-connect
Data sharing can provide availability • NAS file locking can ensure integrity for server farms
• Data sharing in SANs is expensive and complex
![Page 27: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/27.jpg)
The Fine Print in Scalability
Maxed-out NAS requires additional mount
points• Users and application configurations must change
SANs can use virtualization• Does “V” ensure integrity?
• Is “V” recoverable? (will it support serverless
backup?)
File system data structures must be
included for non-stop scalability!Secret
![Page 28: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/28.jpg)
StoringStoring
StoringStoring
FilingFiling
The Fine Print in Scalability
How does the filing function
know about the new
storing address
space?
![Page 29: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/29.jpg)
Evolving Architectures
![Page 30: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/30.jpg)
Evolving Storage Network Architectures
Filing technologies are intimately involved
with:• Integrity
• Recoverability
• Security
• Availability (sharing)
• Scalability
• Performance (efficiency - working smart)
Hmmmmm? Maybe filing could be important!
Good Idea
![Page 31: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/31.jpg)
DAFS
Network Appliance (and others?)
Puts rigor into NAS wiring
Important proof of concept for NAS• Eliminate doubts about NAS for database I/O
![Page 32: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/32.jpg)
Relocating Filing Functions in the Storage Network
Systems need to have a data view• Can be achieved by aggregating views from
distributed
Metadata is moved out of host systems• Distributed or network-central
Data structures can be managed by the
subsystem• Enables subsystem-based recoverability
• Delivers enormous scalability
![Page 33: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/33.jpg)
Aggregating Data Views
Virtualization at the file level
Data views from many subsystems are merged
![Page 34: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/34.jpg)
Independence of Metadata from Hosts
Integrity (locking)
Scalability
Metadata
Systems
Storage
12
3
![Page 35: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/35.jpg)
Independence of Metadata from Hosts
Systems
Storage
Distributed Metadata
![Page 36: Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks](https://reader030.vdocument.in/reader030/viewer/2022032518/56649cc55503460f9498f388/html5/thumbnails/36.jpg)
Subsystem-Managed Data Structures
Systemwithout data structure function
Data structure is distributed across multiple subsystems