section 2 – storage systems architecture
DESCRIPTION
Section 2 – Storage Systems Architecture. Introduction. 本章目标及内容. 本章主要介绍存储架构的各个组成部分。首先介绍主机与存储的连接关系,然后从磁盘的基本概念出发,介绍了磁盘阵列和磁盘的 RAID 数据保护等概念。最后,本章介绍了磁盘存储系统的系统结构并深入说明了磁盘存储系统是如何高效的应用于系统环境的。 本章内容包括 5 个方面: 2.1 主机环境( The Host Environment ) 2.2 连接( Connectivity ) - PowerPoint PPT PresentationTRANSCRIPT
© 2006 EMC Corporation. All rights reserved.
Section 2 – Storage Systems ArchitectureSection 2 – Storage Systems Architecture
Introduction
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 2
本章目标及内容 本章主要介绍存储架构的各个组成部分。首先介绍主机与存储的连接关系,然后从磁盘的基本概念出发,介绍了磁盘阵列和磁盘的 RAID数据保护等概念。最后,本章介绍了磁盘存储系统的系统结构并深入说明了磁盘存储系统是如何高效的应用于系统环境的。
本章内容包括 5个方面:2.1 主机环境( The Host Environment)2.2 连接( Connectivity)2.3 物理磁盘( Physical Disks)2.4 磁盘阵列( Disk Arrays)2.5 磁盘存储系统( Disk Storage Systems)
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 3
Section Objectives
Upon completion of this Section, you will be able to:
Describe the host environment.
Describe common connectivity components and protocols.
Describe features of intelligent disk subsystems.
Describe data flow from the host to/from the disk.
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 4
In this Section …
This section contains the following modules:
1. The Host Environment
2. Connectivity
3. Physical Disks
4. RAID Arrays
5. Storage Systems
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 5
Storage System Environment
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 6
Parts of a Storage Environment: Host
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 7
Parts of a Storage Environment: Connectivity
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 8
Parts of a Storage Environment: Storage
© 2006 EMC Corporation. All rights reserved.
The Host Environment The Host Environment
Module 2.1
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 10
The Host Environment
Upon completion of this module, you will be able to:
List the hardware and software components of the host environment
Describe key protocols and concepts used by each component
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 11
Examples of Hosts
Laptop
Server
Group of Servers
Mainframe
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 12
Host Physical Components
Bus
I/O Devices
CPU Storage
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 13
CPU
CPU
Bus
BusALU
Registers
L1 Cache
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 14
Storage
…
0
1
2
3
n
Data 0
Data n
Data 2
Data 3
Data 1
Address Content
Disk
Memory
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 15
Storage Hierarchy – Speed and Cost
Speed
Slow
Fast
CostHighLow
TapeOptical
disk
Magnetic disk
RAM
L2 cache L1 cache
CPU registers
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 16
I/O Devices
Human interface– Keyboard
– Mouse
– Monitor
Computer-computer interface– Network Interface Card (NIC)
Computer-peripheral interface– USB (Universal Serial Bus) port
– Host Bus Adapter (HBA)
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 17
Host Environment: Logical Components
Host
Apps
Volume Management
DBMS Mgmt Utilities
File System
Multi-pathing Software
Device Drivers
HBA HBA HBA
Operating System
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 18
Host
Apps
Volume Management
DBMS Mgmt Utilities
File System
Multi-pathing Software
Device Drivers
HBA HBA HBA
Operating System
File Systems
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 19
File System: Metadata Examples
UNIX (UFS)
File type and permissions
Number of links
Owner and group IDs
Number of bytes in the file
Last file access
Last file modification
Windows (NTFS)
Time stamp and link count
File name
Access rights
File data
Index information
Volume information
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 20
File Systems: Journaling and Logging
Improves data integrity and system restart time over non-journaling file systems.
Uses a separate area called a log or journal.– May hold all data to be written
– May hold only metadata
Disadvantage - slower than other file systems.– Each file system update requires at least 1 extra write – to the
log
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 21
Volume Management
Host
Apps
Volume Management
DBMS Mgmt Utilities
File System
Multi-pathing Software
Device Drivers
HBA HBA HBA
Operating System
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 22
HBAsHost
Apps
Volume Management
DBMS Mgmt Utilities
File System
Multi-pathing Software
Device Drivers
HBA HBA HBA
Operating System
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 23
Improving Data Availability at the Host
Redundancy:
Multiple HBAs
Multi-pathing software
Clustering
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 24
How Files are Moved to and from Storage
Teacher
Configures / Manages
File System Files
Mapped by file system
to
Course File(s)
Reside in
File System Blocks
Disk Physical Extents
Consisting of
LVM Logical Extents
Residing in
Mapped by LVM to
Disk Sectors
Managed by Disk Storage Subsystem
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 25
Module Summary
Key points covered in this module:
Hosts typically have:– Hardware: CPU, memory, buses, disks, ports, and interfaces.
– Software: applications, operating systems, file systems, device drivers, volume managers
HBAs connect hosts to storage devices.
Multi-pathing software uses redundant paths to ensure uninterrupted communication between the host and the storage
Clustering uses redundant host systems to improve data availability
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 26
Check Your Knowledge
What are some examples of hosts?
Describe the hardware components found in most hosts.
What is the function of the operating system?
What is the function of the file system?
What are some techniques that can be used to improve availability at the host?
What is volume management?
© 2006 EMC Corporation. All rights reserved.
ConnectivityConnectivity
Module 2.2
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 28
Connectivity
Upon completion of this module, you will be able to:
Describe the physical components of a networked storage environment.
Describe the logical components (communication protocols) of a networked storage environment.
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 29
Physical Components – Host with Internal Storage
Bus
Disk
Cable
Host
Port
Port
HBA
CPU
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 30
Bus Technology
Serial
Serial Bi-directional
Parallel
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 31
Bus Technology
System Bus – connects CPU to Memory
Local (I/O) Bus – carries data to/from peripheral devices.
Bus width measured in bits
Bus speed measured in MHz
Throughput measured in MB/S
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 32
Connectivity Protocols
Protocol = a defined format for communication – allows the sending and receiving devices to agree on what is being communicated.
Tightly ConnectedEntities
DirectlyAttachedEntities
Network Connected
Entities
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 33
Communication Protocols
Host
Apps
Operating System
PCI
SCSI or IDE/ATA Device Drivers
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 34
Bus Technology - PCI
Peripheral Component Interconnect (PCI) defines the local bus system within a computer
It is an interconnection between microprocessor and attached devices, in which expansion slots are spaced closely for high-speed operation.
Has Plug and Play functionality.
PCI is 32/64 bit
Throughput is 133 MB/sec
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 35
IDE/ATA
Integrated Device Electronics (IDE) / Advanced Technology Attachment (ATA)
Most popular interface used with modern hard disks
Good performance at low cost
Desktop and laptop systems
Inexpensive storage interconnect
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 36
SCSI - Small Computer System Interface
Most popular hard disk interface for servers.
Higher cost than IDE/ATA.
Supports multiple simultaneous data access.
Currently both parallel and serial forms.
Used primarily in “higher end” environments.
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 37
SCSI Model
Target
Initiator
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 38
SCSI Model
Target ID
Initiator ID
LUNs
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 39
SCSI Addressing
Initiator ID - a number from 0 to 15 with the most common value being 7.
Target ID - a number from 0 to 15 LUN - a number that specifies a device addressable
through a target.
Initiator ID Target ID LUN
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 40
Disk Identifier - Addressing
c0 - ControllerInitiator, HBA
PeripheralController
t0
Target
LUNs
d0 d1 d2
Host Addressing– Controller
– Target
– LUN
c0 t0 d0
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 41
SCSI - Pros and Cons
Pros:– Fast transfer speeds, up to 320
megabytes per second
– Reliable, durable components
– Can connect many devices with a single bus, more than just HDs
– SCSI host cards can be put in almost any system
– Full backwards compatibility
Cons:– Configuration and setup
specific to one computer
– Unlike IDE, few BIOS support the standard
– Overwhelming number of variations in the standard, hardware, and connectors
– No common software interfaces and protocol
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 42
Comparison IDE/ATA vs SCSI
Feature IDE/ATA SCSI
Connectivity Market Internal Storage Internal and External Storage
Speed (MB/sec) 100/133/150 320
Hot Pluggable No Yes
Expandability Easier to set up Very good but veryexpensive to set up
Cost/Performance Good High cost/Fasttransfer speed
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 43
Physical Components – Host with External Storage
Bus
Disk
Cable
Host
Port
Port
HBA
CPU
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 44
Fibre Channel
Fibre Channel
Storage Arrays
Host
AppsDBMS Mgmt Utils
File System
LVM
Multipathing Software
Device Drivers
HBA HBA HBA
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 45
External Storage Interfaces – A Comparison
SCSI– Limited distance
– Limited device count
– Usually limited to single initiator
– Single-ported drives
Fibre Channel– Greater distance
– High device count in SANs
– Multiple initiators
– Dual-ported drives
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 46
Fibre Channel Connectivity
Switches Storage
Hosts
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 47
Module Summary
Key points covered in this module:
The physical components of a networked storage environment.
The logical components (communication protocols) of a networked storage environment.
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 48
Check Your Knowledge
What are the key physical connectivity components of a small systems environment?
What are the key physical connectivity components of networked storage computing environments?
What are the key logical connectivity protocols found in all computing environments?
© 2006 EMC Corporation. All rights reserved.
Physical DisksPhysical Disks
Module 2.3
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 50
Physical Disks
After completing this module, you will be able to:
Describe the major physical components of a disk drive and their function
Define the logical constructs of a physical disk
Describe the access characteristics for disk drives and their performance implications
Describe the logical partitioning of physical drives
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 51
Lesson: Disk Drive Components
Upon completion of this lesson, you will be able to:
Describe the physical components of a disk drive
Describe the physical structure of a disk drive platter
Discuss how the geometry of a disk impacts how data is recorded on a platter
Differentiate between the logical organization of data and the physical organization on a disk drive
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 52
Disk Drive Components: Platters
0011010011101010101000110100111010101010
10110101011010101010
01010100111010101010
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 53
Disk Drive Components: Spindle
Spindle
Platters
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 54
Disk Drive Components: Read/Write Heads
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 55
Disk Drive Components: Actuator
Actuator
Spindle
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 56
Physical Disk Structures: Actuator Arm Assembly
Actuator
R/W Head
R/W Head
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 57
Disk Drive Components: Controller
Bottom View of Disk Drive
HDA
Controller
Interface
Power Connector
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 58
Physical Disk Structures: Sectors and Tracks
Sector
Track
Platter
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 59
Platter Geometry and Zoned-Bit Recording
Platter Without Zones
Sector
Track
Platter With Zones
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 60
Physical Disk Structures: Cylinders
Cylinder
Tracks, Cylinders and Sectors
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 61
Logical Block Addressing
Physical Address = CHS Logical Block Address = Block #
Sector
CylinderHead
Block 0
Block 16
Block 32
Block 48
Block 8
(lower surface)
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 62
What the Host Sees
A
One Logical VolumeMultiple Logical Volumes
ABC
D
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 63
Lesson Summary
Key points covered in this lesson:
Physical drives are made up of:– HDA
Platters connected via a spindle Read/write heads which are positioned by an actuator
– Controller Controls power, communication, positioning, and optimization
Data is structured on a drive using tracks, sectors, and cylinders
The geometry of a disk impacts how data is recorded on a platter
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 64
Lesson: Disk Drive Performance
Upon completion of this lesson, you will be able to:
Describe the factors that impact the performance of a drive
Describe how drive reliability is measured
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 65
Disk Drive Performance: Positioning
Seek time is the time for read/write heads to move between tracks
Seek time specifications include:
– Full stroke
– Average
– Track-to-track
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 66
Disk Drive Performance: Rotational Speed/Latency
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 67
Disk Drive Performance: Command Queuing
Request 1
Request 2
Request 3
Request 4
1234
Request 1
Request 2
Request 3
Request 4
1324
Without Command Queuing
With Command Queuing
1
2
34
1
2
34
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 68
Disk Drive Performance: Data Transfer Rate
InterfaceInterface BufferBufferHBAHBA
Disk Drive
Internal transfer rate measured here
External transfer rate measured here
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 69
Drive Reliability: MTBF
Mean Time Between Failure
Amount of time that one can anticipate a device to work before an incapacitating malfunction occurs– Based on averages
– Measured in hours
Determined by artificially aging the product
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 70
Lesson Summary
Key points covered in this lesson:
Drive performance is impacted by a number of factors including:– Seek time
– Rotational latency
– Command queuing
– Data transfer rate
Drive reliability is measured using MTBF
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 71
Module Summary
Key points covered in this module:
Physical drives are made up of a number of components– HDA – houses the platters, spindles, actuator assemblies (which
include the actuator and the read/write heads)
– Controller - Controls power, communication, positioning, and optimization
Data is structured on a drive using tracks, sectors, and cylinders
Drive performance is impacted by seek time, rotational latency, command queuing, and data transfer rate
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 72
Check Your Knowledge
Describe the purpose of the actuator, the read/write head, and the controller on a drive.
What is the difference between a track, a sector, and a cylinder?
Why is zoned-bit recording used?
What is the difference between seek time and rotational latency?
What is the difference between internal and external data transfer rates?
What purpose does the MTBF specification serve?
© 2006 EMC Corporation. All rights reserved.
RAID ArraysRAID Arrays
Module 2.4
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 74
RAID Arrays
After completing this module, you will be able to:
Describe what RAID is and the needs it addresses
Describe the concepts upon which RAID is built
Compare and contrast common RAID levels
Recommend the use of the common RAID levels based on performance and availability considerations
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 75
RAID - Redundant Array of Independent Disks
RAIDController
RAIDController
RAID Array
Host
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 76
RAID Components
RAIDController
RAIDController
Logical Array
Logical Array
Physical Array
RAID Array
Host
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 77
Data Organization: Strips and Stripes
Stripe 1Stripe 2Stripe 3
Strips
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 78
RAID Performance: Striping
Logical Array
LUNRAIDController
RAIDController
Host
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 79
RAID Redundancy: Mirroring
RAID Array
Mirrored Disk
RAIDController
RAIDController
Host
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 80
RAID Redundancy: Parity
Parity Disk
0
84
1
95
2
106
3
117
0 1 2 3
8 9 10 114 5 6 7
RAIDController
RAIDController
Host
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 81
Parity Calculation
Parity
Data
Data
Data
Data
4
2
3
5
14
5 + 3 + 4 + 2 = 14
The middle drive fails:
5 + 3 + ? + 2 = 14
? = 14 – 5 – 3 – 2
? = 4
RAID Array
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 82
RAID Levels
0 Striped Array with No Fault Tolerance
1 Disk Mirroring
3 Parallel Access Array with Dedicated Parity Disk
4 Striped Array with Independent Disks and a Dedicated Parity Disk
5 Striped Array with Independent Disks and Distributed Parity
Combinations of levels (I.e., 1 + 0, 0 + 1, etc.)
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 83
RAID 0 – Striped Array with no Fault Tolerance
RAIDController
RAIDControllerBlock 4Block 4 Block 4Block 4Block 3Block 3 Block 3Block 3Block 2Block 2 Block 2Block 2Block 1Block 1 Block 1Block 1Block 0Block 0 Block 0Block 0
Host
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 84
RAID 1 – Disk Mirroring
RAIDController
RAIDControllerBlock 1Block 1 Block 1Block 1Block 1Block 1Block 0Block 0 Block 0Block 0Block 0Block 0
Host
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 85
RAID 0+1 – Striping and Mirroring
RAIDController
RAIDControllerBlock 3Block 3 Block 3Block 3Block 3Block 3Block 2Block 2 Block 2Block 2Block 2Block 2Block 1Block 1 Block 1Block 1Block 1Block 1Block 0Block 0 Block 0Block 0Block 0Block 0
Host
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 86
RAID 1+0 – Mirroring and Striping
RAIDController
RAIDControllerBlock 3Block 3 Block 3Block 3Block 3Block 3Block 2Block 2 Block 2Block 2Block 2Block 2Block 1Block 1 Block 1Block 1Block 1Block 1Block 0Block 0 Block 0Block 0Block 0Block 0
Host
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 87
RAID 0+1 vs. RAID 1+0
Benefits are identical under normal operations.
Rebuild operations are very different.– RAID 1+0 uses a mirrored pair – only 1 disk is rebuilt if a disk fails
– RAID 0+1 if a single drive fails, the entire stripe is faulted RAID is 0+1 is a poorer solution and is less common
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 88
RAID 3 - Parallel Transfer with Dedicated Parity Disk
RAIDController
RAIDController
Block 1Block 1
Block 2Block 2
Block 3Block 3
P 0 1 2 3
Block 0Block 0Block 3Block 3Block 2Block 2Block 1Block 1Block 0Block 0
ParityGenerated
Host
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 89
RAID 4 - Striping with Dedicated Parity Disk
RAIDController
RAIDController
P 0 1 2 3
Block 0Block 0
Block 0Block 0
Block 4Block 4
Block 1Block 1
Block 5Block 5
Block 2Block 2
Block 6Block 6
Block 3Block 3
Block 7Block 7
P 0 1 2 3P 0 1 2 3
P 4 5 6 7P 4 5 6 7
ParityGenerated
Block 0Block 0
P 0 1 2 3P 0 1 2 3
Host
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 90
Block 0Block 0
P 0 1 2 3P 0 1 2 3
RAID 5 - Independent Disks with Distributed Parity
Block 7Block 7
RAIDController
RAIDController
P 0 1 2 3
Block 0Block 4Block 0
Block 1Block 1
Block 5Block 5
Block 2Block 2
Block 6Block 6
Block 3Block 3
ParityGenerated
Block 0Block 0
P 0 1 2 3P 0 1 2 3
Block 4Block 4
P 4 5 6 7P 4 5 6 7P 4 5 6 7P 4 5 6 7
Block 4Block 4
P 4 5 6 7
Block 4ParityGenerated
Host
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 91
RAID Implementations
Hardware (usually a specialized disk controller card)– Controls all drives that are attached it
– Performs all RAID-related functions including volume management
– Array(s) appear to the host operating system as a regular disk drive
– Dedicated cache to improve performance
– Generally provides some type of administrative software
Software – Generally runs as part of the operating system
– Volume management and performed by the server
– Provides more flexibility for hardware, which can reduce the cost
– Performance is dependent on CPU load & server performance
– Has limited functionality
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 92
Hot Spares
RAIDController
RAIDController
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 93
Hot Swap
RAIDController
RAIDController
RAIDController
RAIDController
RAIDController
RAIDController
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 94
Module Summary
Key points covered in this module:
What RAID is and the needs it addresses
The concepts upon which RAID is built
The difference between RAID levels
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 95
Check Your Knowledge
What is a RAID array?
What benefits do RAID arrays provide?
What methods can be used to provide higher data availability in a RAID array?
What is the primary difference between RAID 3 and RAID 5?
Why might you use a combined RAID level, such as RAID 1+0 or 0+1?
© 2006 EMC Corporation. All rights reserved.
Disk Storage SystemsDisk Storage Systems
Module 2.5
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 97
Disk Storage Systems
After completing this module, you will be able to:
Describe the components of an intelligent storage system
Describe the configuration of a logical disk
Discuss the methods employed to ensure that a host can access a storage volume
Discuss back end volume protection
Discuss front end host configuration
Describe the I/O flow from the back end to the physical disks
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 98
Lesson: Intelligent Storage System Overview
After completing this lesson, you will be able to:
List the benefits of intelligent storage systems
Compare and contrast integrated and modular approaches to intelligent storage systems
Describe the I/O flow through the storage system
Describe the logical elements of an intelligent storage system
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 99
What is an Intelligent Storage System?
A disk storage system which distributes data over several devices and manages access to that data.
When implemented properly, it provides the following benefits over individual storage devices:– Increased capacity
– Improved performance
– Easier data management
– Better data availability
– More robust backup/restore capabilities
– Improved flexibility and scalability
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 100
Monolithic (Integrated) Storage Systems
Monolithic
FC Ports
Port Processors
Cache
RAID Controllers
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 101
Modular Storage Systems
Rack
Servers
Disk Modules
Control Modulewith Disks
FC Switches
Modular
Host Interface
Cache
RAID
Controller AController A
Host Interface
Cache
RAID
Controller BController B
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 102
Elements in an Intelligent Storage System
Intelligent Storage System
CacheCache
Front End Back End
Cache
Physical Disks
Host Connectivity
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 103
Intelligent Storage System: Front End
Note: Include redundancy in the channels to and from the ports.
Intelligent Storage System
Ports
Host Connectivity
Controllers
Front End Back End
Cache
Physical Disks
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 104
Front End Command Queuing
FRONT
END
FRONT
END
Request 1
Request 2
Request 3
Request 4
1234
FRONT
END
FRONT
END
Request 1
Request 2
Request 3
Request 4
1324
Without Command Queuing
With Command Queuing
1
2
34
1
2
34
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 105
Intelligent Storage System: Cache
Intelligent Storage System
Host Connectivity
Front End Back End
Cache
Physical Disks
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 106
Intelligent Storage System: Back End
Host Connectivity
PortsControllers
Front End Back End
Cache
Physical Disks
Intelligent Storage System
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 107
Intelligent Storage System: Physical Disks
Host Connectivity
Front End Back End
Cache
Physical Disks
Intelligent Storage System
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 108
I/O Example: Read RequestsI/O Example: Read Requests
Intelligent Storage System
Host Connectivity
Front End Back End
Cache
Physical Disks
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 109
I/O Example: Write Requests
Intelligent Storage System
Host Connectivity
Front End Back End
Cache
Physical Disks
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 110
What the Host Sees
Intelligent Storage System
LUN 0LUN 1LUN 2
LUN 0
LUN 1
LUN 2
Host
Host
Back EndPhysical Disks
Cache
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 111
The Host and Logical Device Names
HostVolumeManager
Host
/dev/rdsk/c1t1d0
/dev/rdsk/c1t1d1
\\.\PhysicalDrive0
VolumeManager
Intelligent Storage System
LUN 0LUN 1LUN 2
LUN 0
LUN 1
LUN 2
Back EndPhysical Disks
Cache
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 112
Disk Organization in a Storage System
Intelligent Storage System
LUN 0
LUN 1
Host
Host
LUN 0
LUN 1
Back End Physical Disks
Cache
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 113
Lesson Summary
Key points covered in this lesson:
An intelligent disk storage system:– Distributes data over several devices and manages access to that
data
– Has a front end, cache, a back end, and physical disks.
– Use the virtual disks to provide optimal performance and capacity.
– Individual disks within a RAID set can be divided into logical units.
– The same concept can be applied to entire RAID sets.
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 114
Lesson: Cache – A Closer Look
After completing this lesson, you will be able to:
Define cache
Distinguish between multipurpose cache and configurable cache
Describe cache hits and misses
Describe algorithms to manage cache
Trace the I/O flow from the cache to the back end to the physical disks
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 115
What is Cache in a Storage System
A memory space used by a disk storage system to reduce the time required to read data/write data. It is usually made from very fast memory
CacheRead
RequestWrite
Request
Acknowledgment
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 116
How Cache is Structured
Data Store
Tag RAM
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 117
ReadRequest
Read Cache ‘Hits’ and ‘Misses’
CacheRead
Request
Cache
Data found in cache = ‘Hit’
No data found = ‘Miss’
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 118
Algorithms Used to Manage Read Cache
Least Recently Used (LRU) – Determines which items are
accessed frequently/infrequently
– Discards least recently used data
Read Ahead (pre-fetch)– Accesses data sequentially and puts
it into cache before it is requested
– May assume that data recently accessed will not be needed again.
New Data
Oldest Data
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 119
WriteRequest
Write Algorithms
WriteRequest
Write-through Cache
Write-back
Acknowledgement
Acknowledgement
Cache
Cache
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 120
Write Cache: Performance
Manage peak I/O requests “bursts” through flushing– Least-recently used pages are flushed from cache to the drives
For maximum performance:– Provide headroom in write cache for I/O bursts
Coalesce small host writes into larger disk writes– Improve sequentiality at the disk
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 121
Lesson Summary
Key points covered in this lesson:
Cache is a memory space used by a disk storage system to reduce the time required to read data/write data.
It can speed up both read and write operations.
Cache read algorithms include:– Least Recently Used (LRU)
– Read Ahead (pre-fetch)
Cache write algorithms include:– Write-through
– Write-back
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 122
Module Summary
Key points covered in this module:
An intelligent disk storage system distributes data over several devices and manages access to that data.
Monolithic storage systems are generally aimed at the enterprise level, centralizing data in a powerful system with hundreds of drives.
Modular storage systems provide storage to a smaller number of (typically) Windows or Unix servers than larger integrated storage systems.
Cache is an important part of intelligent disk storage systems as it can be used to improve performance.
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 123
Check Your Knowledge
What are the parts of an Intelligent Disk Subsystem?
What is the difference between a monolithic and a modular array?
What is the difference between cache hit and a cache miss?
What is the difference between Least Recently Used and Read Ahead cache?
What is the difference between Write-through and Write-back cache?
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 124
Apply Your Knowledge
Upon completion of this case study, you will be able to:
Describe the basic architecture of the CLARiiON modular storage array.
Describe the basic architecture of the Symmetrix integrated storage array.
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 125
CLARiiON CX3-80 Architecture
Power supply
Power supply
Fan FanFan
SPS
Up to 480 drives max per storage system (CX3-80)
4Gb/s LCC
4Gb/s LCC
4Gb/s LCC
4Gb/s LCC
4Gb/s LCC
4Gb/s LCC
4Gb/s LCC
4Gb/s LCC
UltraScaleStorage Processor
UltraScaleStorage Processor
Fibre Channel
Mirrored cache
Fibre Channel
CPU
Mirrored cache
CPU
FC FC
CPU
FC
CPU
FCFC FC FCFC
Fan
2/4 Gb/s Fibre Channel Back End
2/4 Gb/s Fibre Channel Back End
1/2/4 Gb/s Fibre Channel Front End
CLARiiON Messaging Interface (CMI)
Multi-Lane PCI-Express bridge link
SPS
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 126
Assigning CLARiiON LUNs to Hosts
CLARiiON disks are grouped into RAID Groups– Disks from any enclosure may be used in a RAID Group
– All disks in a RAID Group must be either Fibre Channel or ATA
– A RAID Group is the ‘RAID set’ discussed earlier
– A RAID Group may be a single disk, or RAID Level 0, 1, 1/0, 3 or 5
The RAID Group is then partitioned into LUNs– All LUNs in a RAID Group will be the same RAID Level
The LUNs are then made accessible to hosts– CLARiiON-resident software ensures that LUNs are seen only by the
hosts that own them
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 127
EMC Symmetrix DMX Array
Direct Matrix Interconnect
Dynamic Global Memory
Enginuity Operating Environment
Processing Power
Flexible Back-End Configurations
Fault-tolerant Design
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 128
Symmetrix DMX Series Direct Matrix Architecture
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 129
Symmetrix DMX: Dual-ported Disk and Redundant Directors
Disk Director 1 Disk Director 16
P
S
P
S
P
S
P
S
S
P
S
P
S
P
S
P
P = Primary Connection to DriveS= Secondary Connection for Redundancy
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 130
Configuring Symmetrix Logical Volumes (SLV)
Initial configuration of Symmetrix Logical Volumes is done via the Symmetrix Service Processor and the SymmWin interface/application– A configuration file (IMPL.BIN) is created and loaded on to the array
Subsequent configuration changes can be performed online using EMC ControlCenter (GUI) or by using Solutions Enabler (CLI)
Physical Disk
Physical Disk
Physical Disk
Physical Disk
Physical Disk Symmetrix Service Processor
Running SymmWin Application
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 131
RAID1 – Symmetrix Logical Volume
RAID1 SLV– Data is written to two hyper volumes on two different physical disks
which are accessed via two different disk directors
Host is unaware of data protection being applied
Physical Drive
LV 04B M2
Different Disk Director
Physical Drive
LV 04B M1
Disk Director
Logical Volume 04B
Host AddressTarget = 1LUN = 0
Hyper
Volumes
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 132
Data Protection
Mirroring (RAID 1) – Highest performance, availability and functionality
– Two hyper mirrors form one Symmetrix Logical Volume located on separate physical drives
Parity RAID (Not available on DMX3)– 3 +1 (3 data and 1 parity volume) or 7 +1 (7 data and 1 parity volume)
Raid 5 Striped RAID volumes– Data blocks are striped horizontally across the members of the RAID group
( 4 or 8 member group); parity blocks rotate among the group members
RAID 10 Mirrored Striped Mainframe Volumes
Dynamic Sparing
SRDF (Symmetrix Remote Data Facility)– Mirror of Symmetrix logical Volume maintained in a separate Symmetrix
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 133
Assigning Symmetrix Logical Volumes to Hosts
Configure Symmetrix Logical Volumes
Map Symmetrix Logical Volumes to Front-end ports– Performed via EMC ControlCenter or Solutions Enabler
Make Symmetrix Logical Volumes accessible to hosts– SAN Environment
Zone Hosts to Front-end ports Perform LUN Masking
Can be performed via EMC ControlCenter or Solutions Enabler LUN Masking information is maintained on the Symmetrix in the VCM Database
(VCMDB)■ LUN Masking information is also flashed to all the front-end directors
© 2006 EMC Corporation. All rights reserved.
Data FlowData Flow
Exercise
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 135
Q: Architecture ExerciseIdentify the components of a data storage environment:
C
A
ED
G
B F
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 136
Q: Data Flow Exercise – Write Cache HitIn this example, the storage system uses write-back cache.
Identify the operations performed when writing data to disk, then put them in the correct order:
32
4
6
5
1
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 137
Q: Data Flow Exercise – Read Cache HitThe storage system uses write-back cache. Identify the operations performed when reading data from disk. They are presented in the correct order (1 to 4):
34
12
© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 138
Q: Data Flow Exercise – Read Cache MissThe storage system uses write-back cache. Identify the operations
performed when reading data, then put them in the correct order:
3
6 1
5
2
4