section 2 – storage systems architecture

138
© 2006 EMC Corporation. All rights reserved. Section 2 – Storage Systems Architecture Introduction

Upload: lisle

Post on 12-Jan-2016

36 views

Category:

Documents


5 download

DESCRIPTION

Section 2 – Storage Systems Architecture. Introduction. 本章目标及内容. 本章主要介绍存储架构的各个组成部分。首先介绍主机与存储的连接关系,然后从磁盘的基本概念出发,介绍了磁盘阵列和磁盘的 RAID 数据保护等概念。最后,本章介绍了磁盘存储系统的系统结构并深入说明了磁盘存储系统是如何高效的应用于系统环境的。 本章内容包括 5 个方面: 2.1 主机环境( The Host Environment ) 2.2 连接( Connectivity ) - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved.

Section 2 – Storage Systems ArchitectureSection 2 – Storage Systems Architecture

Introduction

Page 2: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 2

本章目标及内容 本章主要介绍存储架构的各个组成部分。首先介绍主机与存储的连接关系,然后从磁盘的基本概念出发,介绍了磁盘阵列和磁盘的 RAID数据保护等概念。最后,本章介绍了磁盘存储系统的系统结构并深入说明了磁盘存储系统是如何高效的应用于系统环境的。

本章内容包括 5个方面:2.1 主机环境( The Host Environment)2.2 连接( Connectivity)2.3 物理磁盘( Physical Disks)2.4 磁盘阵列( Disk Arrays)2.5 磁盘存储系统( Disk Storage Systems)

Page 3: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 3

Section Objectives

Upon completion of this Section, you will be able to:

Describe the host environment.

Describe common connectivity components and protocols.

Describe features of intelligent disk subsystems.

Describe data flow from the host to/from the disk.

Page 4: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 4

In this Section …

This section contains the following modules:

1. The Host Environment

2. Connectivity

3. Physical Disks

4. RAID Arrays

5. Storage Systems

Page 5: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 5

Storage System Environment

Page 6: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 6

Parts of a Storage Environment: Host

Page 7: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 7

Parts of a Storage Environment: Connectivity

Page 8: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 8

Parts of a Storage Environment: Storage

Page 9: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved.

The Host Environment The Host Environment

Module 2.1

Page 10: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 10

The Host Environment

Upon completion of this module, you will be able to:

List the hardware and software components of the host environment

Describe key protocols and concepts used by each component

Page 11: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 11

Examples of Hosts

Laptop

Server

Group of Servers

Mainframe

Page 12: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 12

Host Physical Components

Bus

I/O Devices

CPU Storage

Page 13: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 13

CPU

CPU

Bus

BusALU

Registers

L1 Cache

Page 14: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 14

Storage

0

1

2

3

n

Data 0

Data n

Data 2

Data 3

Data 1

Address Content

Disk

Memory

Page 15: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 15

Storage Hierarchy – Speed and Cost

Speed

Slow

Fast

CostHighLow

TapeOptical

disk

Magnetic disk

RAM

L2 cache L1 cache

CPU registers

Page 16: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 16

I/O Devices

Human interface– Keyboard

– Mouse

– Monitor

Computer-computer interface– Network Interface Card (NIC)

Computer-peripheral interface– USB (Universal Serial Bus) port

– Host Bus Adapter (HBA)

Page 17: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 17

Host Environment: Logical Components

Host

Apps

Volume Management

DBMS Mgmt Utilities

File System

Multi-pathing Software

Device Drivers

HBA HBA HBA

Operating System

Page 18: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 18

Host

Apps

Volume Management

DBMS Mgmt Utilities

File System

Multi-pathing Software

Device Drivers

HBA HBA HBA

Operating System

File Systems

Page 19: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 19

File System: Metadata Examples

UNIX (UFS)

File type and permissions

Number of links

Owner and group IDs

Number of bytes in the file

Last file access

Last file modification

Windows (NTFS)

Time stamp and link count

File name

Access rights

File data

Index information

Volume information

Page 20: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 20

File Systems: Journaling and Logging

Improves data integrity and system restart time over non-journaling file systems.

Uses a separate area called a log or journal.– May hold all data to be written

– May hold only metadata

Disadvantage - slower than other file systems.– Each file system update requires at least 1 extra write – to the

log

Page 21: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 21

Volume Management

Host

Apps

Volume Management

DBMS Mgmt Utilities

File System

Multi-pathing Software

Device Drivers

HBA HBA HBA

Operating System

Page 22: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 22

HBAsHost

Apps

Volume Management

DBMS Mgmt Utilities

File System

Multi-pathing Software

Device Drivers

HBA HBA HBA

Operating System

Page 23: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 23

Improving Data Availability at the Host

Redundancy:

Multiple HBAs

Multi-pathing software

Clustering

Page 24: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 24

How Files are Moved to and from Storage

Teacher

Configures / Manages

File System Files

Mapped by file system

to

Course File(s)

Reside in

File System Blocks

Disk Physical Extents

Consisting of

LVM Logical Extents

Residing in

Mapped by LVM to

Disk Sectors

Managed by Disk Storage Subsystem

Page 25: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 25

Module Summary

Key points covered in this module:

Hosts typically have:– Hardware: CPU, memory, buses, disks, ports, and interfaces.

– Software: applications, operating systems, file systems, device drivers, volume managers

HBAs connect hosts to storage devices.

Multi-pathing software uses redundant paths to ensure uninterrupted communication between the host and the storage

Clustering uses redundant host systems to improve data availability

Page 26: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 26

Check Your Knowledge

What are some examples of hosts?

Describe the hardware components found in most hosts.

What is the function of the operating system?

What is the function of the file system?

What are some techniques that can be used to improve availability at the host?

What is volume management?

Page 27: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved.

ConnectivityConnectivity

Module 2.2

Page 28: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 28

Connectivity

Upon completion of this module, you will be able to:

Describe the physical components of a networked storage environment.

Describe the logical components (communication protocols) of a networked storage environment.

Page 29: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 29

Physical Components – Host with Internal Storage

Bus

Disk

Cable

Host

Port

Port

HBA

CPU

Page 30: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 30

Bus Technology

Serial

Serial Bi-directional

Parallel

Page 31: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 31

Bus Technology

System Bus – connects CPU to Memory

Local (I/O) Bus – carries data to/from peripheral devices.

Bus width measured in bits

Bus speed measured in MHz

Throughput measured in MB/S

Page 32: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 32

Connectivity Protocols

Protocol = a defined format for communication – allows the sending and receiving devices to agree on what is being communicated.

Tightly ConnectedEntities

DirectlyAttachedEntities

Network Connected

Entities

Page 33: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 33

Communication Protocols

Host

Apps

Operating System

PCI

SCSI or IDE/ATA Device Drivers

Page 34: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 34

Bus Technology - PCI

Peripheral Component Interconnect (PCI) defines the local bus system within a computer

It is an interconnection between microprocessor and attached devices, in which expansion slots are spaced closely for high-speed operation.

Has Plug and Play functionality.

PCI is 32/64 bit

Throughput is 133 MB/sec

Page 35: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 35

IDE/ATA

Integrated Device Electronics (IDE) / Advanced Technology Attachment (ATA)

Most popular interface used with modern hard disks

Good performance at low cost

Desktop and laptop systems

Inexpensive storage interconnect

Page 36: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 36

SCSI - Small Computer System Interface

Most popular hard disk interface for servers.

Higher cost than IDE/ATA.

Supports multiple simultaneous data access.

Currently both parallel and serial forms.

Used primarily in “higher end” environments.

Page 37: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 37

SCSI Model

Target

Initiator

Page 38: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 38

SCSI Model

Target ID

Initiator ID

LUNs

Page 39: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 39

SCSI Addressing

Initiator ID - a number from 0 to 15 with the most common value being 7.

Target ID - a number from 0 to 15 LUN - a number that specifies a device addressable

through a target.

Initiator ID Target ID LUN

Page 40: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 40

Disk Identifier - Addressing

c0 - ControllerInitiator, HBA

PeripheralController

t0

Target

LUNs

d0 d1 d2

Host Addressing– Controller

– Target

– LUN

c0 t0 d0

Page 41: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 41

SCSI - Pros and Cons

Pros:– Fast transfer speeds, up to 320

megabytes per second

– Reliable, durable components

– Can connect many devices with a single bus, more than just HDs

– SCSI host cards can be put in almost any system

– Full backwards compatibility

Cons:– Configuration and setup

specific to one computer

– Unlike IDE, few BIOS support the standard

– Overwhelming number of variations in the standard, hardware, and connectors

– No common software interfaces and protocol

Page 42: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 42

Comparison IDE/ATA vs SCSI

Feature IDE/ATA SCSI

Connectivity Market Internal Storage Internal and External Storage

Speed (MB/sec) 100/133/150 320

Hot Pluggable No Yes

Expandability Easier to set up Very good but veryexpensive to set up

Cost/Performance Good High cost/Fasttransfer speed

Page 43: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 43

Physical Components – Host with External Storage

Bus

Disk

Cable

Host

Port

Port

HBA

CPU

Page 44: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 44

Fibre Channel

Fibre Channel

Storage Arrays

Host

AppsDBMS Mgmt Utils

File System

LVM

Multipathing Software

Device Drivers

HBA HBA HBA

Page 45: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 45

External Storage Interfaces – A Comparison

SCSI– Limited distance

– Limited device count

– Usually limited to single initiator

– Single-ported drives

Fibre Channel– Greater distance

– High device count in SANs

– Multiple initiators

– Dual-ported drives

Page 46: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 46

Fibre Channel Connectivity

Switches Storage

Hosts

Page 47: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 47

Module Summary

Key points covered in this module:

The physical components of a networked storage environment.

The logical components (communication protocols) of a networked storage environment.

Page 48: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 48

Check Your Knowledge

What are the key physical connectivity components of a small systems environment?

What are the key physical connectivity components of networked storage computing environments?

What are the key logical connectivity protocols found in all computing environments?

Page 49: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved.

Physical DisksPhysical Disks

Module 2.3

Page 50: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 50

Physical Disks

After completing this module, you will be able to:

Describe the major physical components of a disk drive and their function

Define the logical constructs of a physical disk

Describe the access characteristics for disk drives and their performance implications

Describe the logical partitioning of physical drives

Page 51: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 51

Lesson: Disk Drive Components

Upon completion of this lesson, you will be able to:

Describe the physical components of a disk drive

Describe the physical structure of a disk drive platter

Discuss how the geometry of a disk impacts how data is recorded on a platter

Differentiate between the logical organization of data and the physical organization on a disk drive

Page 52: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 52

Disk Drive Components: Platters

0011010011101010101000110100111010101010

10110101011010101010

01010100111010101010

Page 53: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 53

Disk Drive Components: Spindle

Spindle

Platters

Page 54: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 54

Disk Drive Components: Read/Write Heads

Page 55: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 55

Disk Drive Components: Actuator

Actuator

Spindle

Page 56: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 56

Physical Disk Structures: Actuator Arm Assembly

Actuator

R/W Head

R/W Head

Page 57: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 57

Disk Drive Components: Controller

Bottom View of Disk Drive

HDA

Controller

Interface

Power Connector

Page 58: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 58

Physical Disk Structures: Sectors and Tracks

Sector

Track

Platter

Page 59: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 59

Platter Geometry and Zoned-Bit Recording

Platter Without Zones

Sector

Track

Platter With Zones

Page 60: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 60

Physical Disk Structures: Cylinders

Cylinder

Tracks, Cylinders and Sectors

Page 61: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 61

Logical Block Addressing

Physical Address = CHS Logical Block Address = Block #

Sector

CylinderHead

Block 0

Block 16

Block 32

Block 48

Block 8

(lower surface)

Page 62: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 62

What the Host Sees

A

One Logical VolumeMultiple Logical Volumes

ABC

D

Page 63: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 63

Lesson Summary

Key points covered in this lesson:

Physical drives are made up of:– HDA

Platters connected via a spindle Read/write heads which are positioned by an actuator

– Controller Controls power, communication, positioning, and optimization

Data is structured on a drive using tracks, sectors, and cylinders

The geometry of a disk impacts how data is recorded on a platter

Page 64: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 64

Lesson: Disk Drive Performance

Upon completion of this lesson, you will be able to:

Describe the factors that impact the performance of a drive

Describe how drive reliability is measured

Page 65: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 65

Disk Drive Performance: Positioning

Seek time is the time for read/write heads to move between tracks

Seek time specifications include:

– Full stroke

– Average

– Track-to-track

Page 66: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 66

Disk Drive Performance: Rotational Speed/Latency

Page 67: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 67

Disk Drive Performance: Command Queuing

Request 1

Request 2

Request 3

Request 4

1234

Request 1

Request 2

Request 3

Request 4

1324

Without Command Queuing

With Command Queuing

1

2

34

1

2

34

Page 68: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 68

Disk Drive Performance: Data Transfer Rate

InterfaceInterface BufferBufferHBAHBA

Disk Drive

Internal transfer rate measured here

External transfer rate measured here

Page 69: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 69

Drive Reliability: MTBF

Mean Time Between Failure

Amount of time that one can anticipate a device to work before an incapacitating malfunction occurs– Based on averages

– Measured in hours

Determined by artificially aging the product

Page 70: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 70

Lesson Summary

Key points covered in this lesson:

Drive performance is impacted by a number of factors including:– Seek time

– Rotational latency

– Command queuing

– Data transfer rate

Drive reliability is measured using MTBF

Page 71: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 71

Module Summary

Key points covered in this module:

Physical drives are made up of a number of components– HDA – houses the platters, spindles, actuator assemblies (which

include the actuator and the read/write heads)

– Controller - Controls power, communication, positioning, and optimization

Data is structured on a drive using tracks, sectors, and cylinders

Drive performance is impacted by seek time, rotational latency, command queuing, and data transfer rate

Page 72: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 72

Check Your Knowledge

Describe the purpose of the actuator, the read/write head, and the controller on a drive.

What is the difference between a track, a sector, and a cylinder?

Why is zoned-bit recording used?

What is the difference between seek time and rotational latency?

What is the difference between internal and external data transfer rates?

What purpose does the MTBF specification serve?

Page 73: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved.

RAID ArraysRAID Arrays

Module 2.4

Page 74: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 74

RAID Arrays

After completing this module, you will be able to:

Describe what RAID is and the needs it addresses

Describe the concepts upon which RAID is built

Compare and contrast common RAID levels

Recommend the use of the common RAID levels based on performance and availability considerations

Page 75: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 75

RAID - Redundant Array of Independent Disks

RAIDController

RAIDController

RAID Array

Host

Page 76: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 76

RAID Components

RAIDController

RAIDController

Logical Array

Logical Array

Physical Array

RAID Array

Host

Page 77: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 77

Data Organization: Strips and Stripes

Stripe 1Stripe 2Stripe 3

Strips

Page 78: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 78

RAID Performance: Striping

Logical Array

LUNRAIDController

RAIDController

Host

Page 79: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 79

RAID Redundancy: Mirroring

RAID Array

Mirrored Disk

RAIDController

RAIDController

Host

Page 80: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 80

RAID Redundancy: Parity

Parity Disk

0

84

1

95

2

106

3

117

0 1 2 3

8 9 10 114 5 6 7

RAIDController

RAIDController

Host

Page 81: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 81

Parity Calculation

Parity

Data

Data

Data

Data

4

2

3

5

14

5 + 3 + 4 + 2 = 14

The middle drive fails:

5 + 3 + ? + 2 = 14

? = 14 – 5 – 3 – 2

? = 4

RAID Array

Page 82: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 82

RAID Levels

0 Striped Array with No Fault Tolerance

1 Disk Mirroring

3 Parallel Access Array with Dedicated Parity Disk

4 Striped Array with Independent Disks and a Dedicated Parity Disk

5 Striped Array with Independent Disks and Distributed Parity

Combinations of levels (I.e., 1 + 0, 0 + 1, etc.)

Page 83: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 83

RAID 0 – Striped Array with no Fault Tolerance

RAIDController

RAIDControllerBlock 4Block 4 Block 4Block 4Block 3Block 3 Block 3Block 3Block 2Block 2 Block 2Block 2Block 1Block 1 Block 1Block 1Block 0Block 0 Block 0Block 0

Host

Page 84: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 84

RAID 1 – Disk Mirroring

RAIDController

RAIDControllerBlock 1Block 1 Block 1Block 1Block 1Block 1Block 0Block 0 Block 0Block 0Block 0Block 0

Host

Page 85: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 85

RAID 0+1 – Striping and Mirroring

RAIDController

RAIDControllerBlock 3Block 3 Block 3Block 3Block 3Block 3Block 2Block 2 Block 2Block 2Block 2Block 2Block 1Block 1 Block 1Block 1Block 1Block 1Block 0Block 0 Block 0Block 0Block 0Block 0

Host

Page 86: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 86

RAID 1+0 – Mirroring and Striping

RAIDController

RAIDControllerBlock 3Block 3 Block 3Block 3Block 3Block 3Block 2Block 2 Block 2Block 2Block 2Block 2Block 1Block 1 Block 1Block 1Block 1Block 1Block 0Block 0 Block 0Block 0Block 0Block 0

Host

Page 87: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 87

RAID 0+1 vs. RAID 1+0

Benefits are identical under normal operations.

Rebuild operations are very different.– RAID 1+0 uses a mirrored pair – only 1 disk is rebuilt if a disk fails

– RAID 0+1 if a single drive fails, the entire stripe is faulted RAID is 0+1 is a poorer solution and is less common

Page 88: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 88

RAID 3 - Parallel Transfer with Dedicated Parity Disk

RAIDController

RAIDController

Block 1Block 1

Block 2Block 2

Block 3Block 3

P 0 1 2 3

Block 0Block 0Block 3Block 3Block 2Block 2Block 1Block 1Block 0Block 0

ParityGenerated

Host

Page 89: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 89

RAID 4 - Striping with Dedicated Parity Disk

RAIDController

RAIDController

P 0 1 2 3

Block 0Block 0

Block 0Block 0

Block 4Block 4

Block 1Block 1

Block 5Block 5

Block 2Block 2

Block 6Block 6

Block 3Block 3

Block 7Block 7

P 0 1 2 3P 0 1 2 3

P 4 5 6 7P 4 5 6 7

ParityGenerated

Block 0Block 0

P 0 1 2 3P 0 1 2 3

Host

Page 90: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 90

Block 0Block 0

P 0 1 2 3P 0 1 2 3

RAID 5 - Independent Disks with Distributed Parity

Block 7Block 7

RAIDController

RAIDController

P 0 1 2 3

Block 0Block 4Block 0

Block 1Block 1

Block 5Block 5

Block 2Block 2

Block 6Block 6

Block 3Block 3

ParityGenerated

Block 0Block 0

P 0 1 2 3P 0 1 2 3

Block 4Block 4

P 4 5 6 7P 4 5 6 7P 4 5 6 7P 4 5 6 7

Block 4Block 4

P 4 5 6 7

Block 4ParityGenerated

Host

Page 91: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 91

RAID Implementations

Hardware (usually a specialized disk controller card)– Controls all drives that are attached it

– Performs all RAID-related functions including volume management

– Array(s) appear to the host operating system as a regular disk drive

– Dedicated cache to improve performance

– Generally provides some type of administrative software

Software – Generally runs as part of the operating system

– Volume management and performed by the server

– Provides more flexibility for hardware, which can reduce the cost

– Performance is dependent on CPU load & server performance

– Has limited functionality

Page 92: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 92

Hot Spares

RAIDController

RAIDController

Page 93: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 93

Hot Swap

RAIDController

RAIDController

RAIDController

RAIDController

RAIDController

RAIDController

Page 94: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 94

Module Summary

Key points covered in this module:

What RAID is and the needs it addresses

The concepts upon which RAID is built

The difference between RAID levels

Page 95: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 95

Check Your Knowledge

What is a RAID array?

What benefits do RAID arrays provide?

What methods can be used to provide higher data availability in a RAID array?

What is the primary difference between RAID 3 and RAID 5?

Why might you use a combined RAID level, such as RAID 1+0 or 0+1?

Page 96: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved.

Disk Storage SystemsDisk Storage Systems

Module 2.5

Page 97: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 97

Disk Storage Systems

After completing this module, you will be able to:

Describe the components of an intelligent storage system

Describe the configuration of a logical disk

Discuss the methods employed to ensure that a host can access a storage volume

Discuss back end volume protection

Discuss front end host configuration

Describe the I/O flow from the back end to the physical disks

Page 98: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 98

Lesson: Intelligent Storage System Overview

After completing this lesson, you will be able to:

List the benefits of intelligent storage systems

Compare and contrast integrated and modular approaches to intelligent storage systems

Describe the I/O flow through the storage system

Describe the logical elements of an intelligent storage system

Page 99: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 99

What is an Intelligent Storage System?

A disk storage system which distributes data over several devices and manages access to that data.

When implemented properly, it provides the following benefits over individual storage devices:– Increased capacity

– Improved performance

– Easier data management

– Better data availability

– More robust backup/restore capabilities

– Improved flexibility and scalability

Page 100: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 100

Monolithic (Integrated) Storage Systems

Monolithic

FC Ports

Port Processors

Cache

RAID Controllers

Page 101: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 101

Modular Storage Systems

Rack

Servers

Disk Modules

Control Modulewith Disks

FC Switches

Modular

Host Interface

Cache

RAID

Controller AController A

Host Interface

Cache

RAID

Controller BController B

Page 102: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 102

Elements in an Intelligent Storage System

Intelligent Storage System

CacheCache

Front End Back End

Cache

Physical Disks

Host Connectivity

Page 103: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 103

Intelligent Storage System: Front End

Note: Include redundancy in the channels to and from the ports.

Intelligent Storage System

Ports

Host Connectivity

Controllers

Front End Back End

Cache

Physical Disks

Page 104: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 104

Front End Command Queuing

FRONT

END

FRONT

END

Request 1

Request 2

Request 3

Request 4

1234

FRONT

END

FRONT

END

Request 1

Request 2

Request 3

Request 4

1324

Without Command Queuing

With Command Queuing

1

2

34

1

2

34

Page 105: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 105

Intelligent Storage System: Cache

Intelligent Storage System

Host Connectivity

Front End Back End

Cache

Physical Disks

Page 106: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 106

Intelligent Storage System: Back End

Host Connectivity

PortsControllers

Front End Back End

Cache

Physical Disks

Intelligent Storage System

Page 107: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 107

Intelligent Storage System: Physical Disks

Host Connectivity

Front End Back End

Cache

Physical Disks

Intelligent Storage System

Page 108: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 108

I/O Example: Read RequestsI/O Example: Read Requests

Intelligent Storage System

Host Connectivity

Front End Back End

Cache

Physical Disks

Page 109: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 109

I/O Example: Write Requests

Intelligent Storage System

Host Connectivity

Front End Back End

Cache

Physical Disks

Page 110: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 110

What the Host Sees

Intelligent Storage System

LUN 0LUN 1LUN 2

LUN 0

LUN 1

LUN 2

Host

Host

Back EndPhysical Disks

Cache

Page 111: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 111

The Host and Logical Device Names

HostVolumeManager

Host

/dev/rdsk/c1t1d0

/dev/rdsk/c1t1d1

\\.\PhysicalDrive0

VolumeManager

Intelligent Storage System

LUN 0LUN 1LUN 2

LUN 0

LUN 1

LUN 2

Back EndPhysical Disks

Cache

Page 112: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 112

Disk Organization in a Storage System

Intelligent Storage System

LUN 0

LUN 1

Host

Host

LUN 0

LUN 1

Back End Physical Disks

Cache

Page 113: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 113

Lesson Summary

Key points covered in this lesson:

An intelligent disk storage system:– Distributes data over several devices and manages access to that

data

– Has a front end, cache, a back end, and physical disks.

– Use the virtual disks to provide optimal performance and capacity.

– Individual disks within a RAID set can be divided into logical units.

– The same concept can be applied to entire RAID sets.

Page 114: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 114

Lesson: Cache – A Closer Look

After completing this lesson, you will be able to:

Define cache

Distinguish between multipurpose cache and configurable cache

Describe cache hits and misses

Describe algorithms to manage cache

Trace the I/O flow from the cache to the back end to the physical disks

Page 115: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 115

What is Cache in a Storage System

A memory space used by a disk storage system to reduce the time required to read data/write data. It is usually made from very fast memory

CacheRead

RequestWrite

Request

Acknowledgment

Page 116: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 116

How Cache is Structured

Data Store

Tag RAM

Page 117: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 117

ReadRequest

Read Cache ‘Hits’ and ‘Misses’

CacheRead

Request

Cache

Data found in cache = ‘Hit’

No data found = ‘Miss’

Page 118: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 118

Algorithms Used to Manage Read Cache

Least Recently Used (LRU) – Determines which items are

accessed frequently/infrequently

– Discards least recently used data

Read Ahead (pre-fetch)– Accesses data sequentially and puts

it into cache before it is requested

– May assume that data recently accessed will not be needed again.

New Data

Oldest Data

Page 119: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 119

WriteRequest

Write Algorithms

WriteRequest

Write-through Cache

Write-back

Acknowledgement

Acknowledgement

Cache

Cache

Page 120: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 120

Write Cache: Performance

Manage peak I/O requests “bursts” through flushing– Least-recently used pages are flushed from cache to the drives

For maximum performance:– Provide headroom in write cache for I/O bursts

Coalesce small host writes into larger disk writes– Improve sequentiality at the disk

Page 121: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 121

Lesson Summary

Key points covered in this lesson:

Cache is a memory space used by a disk storage system to reduce the time required to read data/write data.

It can speed up both read and write operations.

Cache read algorithms include:– Least Recently Used (LRU)

– Read Ahead (pre-fetch)

Cache write algorithms include:– Write-through

– Write-back

Page 122: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 122

Module Summary

Key points covered in this module:

An intelligent disk storage system distributes data over several devices and manages access to that data.

Monolithic storage systems are generally aimed at the enterprise level, centralizing data in a powerful system with hundreds of drives.

Modular storage systems provide storage to a smaller number of (typically) Windows or Unix servers than larger integrated storage systems.

Cache is an important part of intelligent disk storage systems as it can be used to improve performance.

Page 123: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 123

Check Your Knowledge

What are the parts of an Intelligent Disk Subsystem?

What is the difference between a monolithic and a modular array?

What is the difference between cache hit and a cache miss?

What is the difference between Least Recently Used and Read Ahead cache?

What is the difference between Write-through and Write-back cache?

Page 124: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 124

Apply Your Knowledge

Upon completion of this case study, you will be able to:

Describe the basic architecture of the CLARiiON modular storage array.

Describe the basic architecture of the Symmetrix integrated storage array.

Page 125: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 125

CLARiiON CX3-80 Architecture

Power supply

Power supply

Fan FanFan

SPS

Up to 480 drives max per storage system (CX3-80)

4Gb/s LCC

4Gb/s LCC

4Gb/s LCC

4Gb/s LCC

4Gb/s LCC

4Gb/s LCC

4Gb/s LCC

4Gb/s LCC

UltraScaleStorage Processor

UltraScaleStorage Processor

Fibre Channel

Mirrored cache

Fibre Channel

CPU

Mirrored cache

CPU

FC FC

CPU

FC

CPU

FCFC FC FCFC

Fan

2/4 Gb/s Fibre Channel Back End

2/4 Gb/s Fibre Channel Back End

1/2/4 Gb/s Fibre Channel Front End

CLARiiON Messaging Interface (CMI)

Multi-Lane PCI-Express bridge link

SPS

Page 126: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 126

Assigning CLARiiON LUNs to Hosts

CLARiiON disks are grouped into RAID Groups– Disks from any enclosure may be used in a RAID Group

– All disks in a RAID Group must be either Fibre Channel or ATA

– A RAID Group is the ‘RAID set’ discussed earlier

– A RAID Group may be a single disk, or RAID Level 0, 1, 1/0, 3 or 5

The RAID Group is then partitioned into LUNs– All LUNs in a RAID Group will be the same RAID Level

The LUNs are then made accessible to hosts– CLARiiON-resident software ensures that LUNs are seen only by the

hosts that own them

Page 127: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 127

EMC Symmetrix DMX Array

Direct Matrix Interconnect

Dynamic Global Memory

Enginuity Operating Environment

Processing Power

Flexible Back-End Configurations

Fault-tolerant Design

Page 128: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 128

Symmetrix DMX Series Direct Matrix Architecture

Page 129: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 129

Symmetrix DMX: Dual-ported Disk and Redundant Directors

Disk Director 1 Disk Director 16

P

S

P

S

P

S

P

S

S

P

S

P

S

P

S

P

P = Primary Connection to DriveS= Secondary Connection for Redundancy

Page 130: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 130

Configuring Symmetrix Logical Volumes (SLV)

Initial configuration of Symmetrix Logical Volumes is done via the Symmetrix Service Processor and the SymmWin interface/application– A configuration file (IMPL.BIN) is created and loaded on to the array

Subsequent configuration changes can be performed online using EMC ControlCenter (GUI) or by using Solutions Enabler (CLI)

Physical Disk

Physical Disk

Physical Disk

Physical Disk

Physical Disk Symmetrix Service Processor

Running SymmWin Application

Page 131: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 131

RAID1 – Symmetrix Logical Volume

RAID1 SLV– Data is written to two hyper volumes on two different physical disks

which are accessed via two different disk directors

Host is unaware of data protection being applied

Physical Drive

LV 04B M2

Different Disk Director

Physical Drive

LV 04B M1

Disk Director

Logical Volume 04B

Host AddressTarget = 1LUN = 0

Hyper

Volumes

Page 132: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 132

Data Protection

Mirroring (RAID 1) – Highest performance, availability and functionality

– Two hyper mirrors form one Symmetrix Logical Volume located on separate physical drives

Parity RAID (Not available on DMX3)– 3 +1 (3 data and 1 parity volume) or 7 +1 (7 data and 1 parity volume)

Raid 5 Striped RAID volumes– Data blocks are striped horizontally across the members of the RAID group

( 4 or 8 member group); parity blocks rotate among the group members

RAID 10 Mirrored Striped Mainframe Volumes

Dynamic Sparing

SRDF (Symmetrix Remote Data Facility)– Mirror of Symmetrix logical Volume maintained in a separate Symmetrix

Page 133: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 133

Assigning Symmetrix Logical Volumes to Hosts

Configure Symmetrix Logical Volumes

Map Symmetrix Logical Volumes to Front-end ports– Performed via EMC ControlCenter or Solutions Enabler

Make Symmetrix Logical Volumes accessible to hosts– SAN Environment

Zone Hosts to Front-end ports Perform LUN Masking

Can be performed via EMC ControlCenter or Solutions Enabler LUN Masking information is maintained on the Symmetrix in the VCM Database

(VCMDB)■ LUN Masking information is also flashed to all the front-end directors

Page 134: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved.

Data FlowData Flow

Exercise

Page 135: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 135

Q: Architecture ExerciseIdentify the components of a data storage environment:

C

A

ED

G

B F

Page 136: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 136

Q: Data Flow Exercise – Write Cache HitIn this example, the storage system uses write-back cache.

Identify the operations performed when writing data to disk, then put them in the correct order:

32

4

6

5

1

Page 137: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 137

Q: Data Flow Exercise – Read Cache HitThe storage system uses write-back cache. Identify the operations performed when reading data from disk. They are presented in the correct order (1 to 4):

34

12

Page 138: Section 2 – Storage Systems Architecture

© 2006 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 138

Q: Data Flow Exercise – Read Cache MissThe storage system uses write-back cache. Identify the operations

performed when reading data, then put them in the correct order:

3

6 1

5

2

4