chapter1-basic storage technology
TRANSCRIPT
-
8/7/2019 Chapter1-Basic Storage Technology
1/48
1Patni Internal 1
-
8/7/2019 Chapter1-Basic Storage Technology
2/48
Copyright 2006 by Patni
2 2
Storage Technology PrimerStorage Technology Primer
-
8/7/2019 Chapter1-Basic Storage Technology
3/48
Copyright 2006 by Patni
3 3
Coverage
Need for Storage & Growth of Data
Why Network Storage?
Storage Performance and Data Protection DAS, NAS, CAS, SAN
Connectivity Technology
Standards Organization
-
8/7/2019 Chapter1-Basic Storage Technology
4/48
Copyright 2006 by Patni
4 4
Need for Storage
-
8/7/2019 Chapter1-Basic Storage Technology
5/48
Copyright 2006 by Patni 5
5
Growth of Data
020
40
60
80
100
120
140
160
180
200
Exabyte
s*
1997 1998 1999 2000 2001 2002 2003 2004 2005 2006
*Exabyte = One Billion Gigabytes
World Wide Production of Information
-
8/7/2019 Chapter1-Basic Storage Technology
6/48
Copyright 2006 by Patni 6
6
Volatile Storage A storage device in which the
contents are lost when power is removed.
Cache Memory -As the microprocessor processes
data, it looks first in the cache memory and if it
finds the data there (from a previous reading of
data), it does not have to do the more time-
consuming reading of data from larger memory. Disk Cache - A disk cache is a mechanism for
improving the time it takes to read from or write
to a hard disk
Volatile Storage
-
8/7/2019 Chapter1-Basic Storage Technology
7/48
Copyright 2006 by Patni 7
7
Non Volatile Storage
MagneticMagnetic OpticalOptical
TapeTape
Mammo h AIT
DLT
LTO
CD
DVD
DiskDisk
ATA Drive
SCSI Drive
Zip Disk
US Disk
ICIC
FFD
SD
MMC
-
8/7/2019 Chapter1-Basic Storage Technology
8/48
Copyright 2006 by Patni 8
8
Storage Issues
-
8/7/2019 Chapter1-Basic Storage Technology
9/48
Copyright 2006 by Patni 9 9
Storage Issues
-
8/7/2019 Chapter1-Basic Storage Technology
10/48
Copyright 2006 by Patni 10 10
Evolution of Storage Management
80s70s Late 90s
Host Centric Client/Server NetworkComputing
Enterprise storage Management
-
8/7/2019 Chapter1-Basic Storage Technology
11/48
Copyright 2006 by Patni 11 11
Types of Network Storage
Direct-attached storage (DAS)Storage devices connected to a server
Network-attached storage (NAS)
Storage devices can be accessed over a computer network
Storage area network (SAN)
Storage devices can be accessed through Fibre Channel
Content Addressable Storage (CAS)
Data stored on multiple intelligent nodes
NAS
IPLAN/WAN
SAN
FibreCha erIP LAN
DAS CAS
IP
-
8/7/2019 Chapter1-Basic Storage Technology
12/48
Copyright 2006 by Patni 12 12
DAS Direc AttachedStorage
Storage hardware that connects to a single
server. Also called: Direct Attached
Storage (DAS). DAS is Local storage and
one-to-one relationship.
Most of storage (about 70%) is connected
via DAS today, but 60% of storage will be
connected via SAN by 2007(IDC)
-
8/7/2019 Chapter1-Basic Storage Technology
13/48
Copyright 2006 by Patni 13 13
DAS (contd)
Advantages
Low capital expense,industry-standard parts
Simple straight forwardsetup
Can run applications andperform as a file server
Relative high-performanceat low-cost within a givenDAS island
Disadvantages
The least flexible of all datasharing topologies
Less thanN
AS file servingperformance
Large administrative effort andhigh cost to maintain
Minimal data sharing
-
8/7/2019 Chapter1-Basic Storage Technology
14/48
Copyright 2006 by Patni 14 14
Network Storage Evolution
Se rver
c
ien ts
Disk drives Server
Disk drives
LAN
DASDAS
NASNAS
SANSAN
-
8/7/2019 Chapter1-Basic Storage Technology
15/48
Copyright 2006 by Patni 15 15
Released in 2002 by EMC
Refers to storage of fixed content, that is data
written once and never changed or cant be
altered in any way. Storage schema was designed specifically for
keeping fixed content secure and in place was
inevitable.
CAS provides a digital fingerprint for a stored
piece of data. Also known as an ID or logical
address.
Content Addressed Storage
-
8/7/2019 Chapter1-Basic Storage Technology
16/48
Copyright 2006 by Patni 16 16
Guaranteed integrity and authenticity of information
Location independent & Long-term retention
Continuous online and available
Automatic distribution across devices
Operation and management of information should be
automated
Singapore context
MAS rule that financial documents must be retained for 6
years Companies Act, all financial records must be maintained
for 7 yearsTelecom need to keep their CDRs for 3 years
Requirement of CAS
-
8/7/2019 Chapter1-Basic Storage Technology
17/48
Copyright 2006 by Patni 17 17
How CAS can Impact Business
Confront the increasingly stringent business and legal
requirements to store information for decades.
Storage ecosystem that can alleviate one of today's growing
storage challenge.
Its core software can manage over decades, extraordinary
amounts of fixed content and to integrate easily with
Independent software vendor applications.
Modification of existing applications via its application
programming interfaces.
-
8/7/2019 Chapter1-Basic Storage Technology
18/48
Copyright 2006 by Patni 18 18
Uses a claim check to address objects.
FCS generates a globally unique identifier (claim
check) to later retrieve the object. The client
presents a token and the server replies with the
object.
The token is a set of ASCII characters. Maybe fixed
in length or varies. Token is a unique string
associated with each object in the system.
How does CAS Work?
-
8/7/2019 Chapter1-Basic Storage Technology
19/48
Copyright 2006 by Patni 19 19
SAN, NAS & CASSAN, NAS & CAS-- C par isonComparison
SAN NAS CAS
Block File Object
High
Performance Mixed
Performance/Capacity
High
Capacity
-
8/7/2019 Chapter1-Basic Storage Technology
20/48
Copyright 2006 by Patni 20 20
RAID
What is RAID?
Stands for Redundant Array of Independent Disks.
Developed to meet the growing demands for data reliability andperformance.
Multiple hard drives are grouped together to form a singlelogical drive.
Why RAID?
Mass storage is successful only with the benefits of this
data protecting scheme. Increases the performance and reliability of data storage
by spreading data across multiple disks.
-
8/7/2019 Chapter1-Basic Storage Technology
21/48
Copyright 2006 by Patni 21 21
RAID Concepts
RAID uses Mirroring, Parity and Striping.
Mirroring
Increases fault tolerance by having two copies of the same data on
separate hard drives.
Downtime is minimal and data recovery is simple.
Increased cost and twice as much as storage.
RAID
Controller
A B C
A B C
Disk1
Disk2
Data(ABC)
-
8/7/2019 Chapter1-Basic Storage Technology
22/48
Copyright 2006 by Patni 22 22
RAID
Controller
A C E
B D F
Disk1
Disk2
Data(ABC DEF)
Striping
Improves performance by distributing data across all drives.
The transfer rates for read and write operations are greatly
increased.
There are two levels of striping:
Byte level striping: breaking up of data into bytes.
Block level striping:breaking up of data into specific block sizes.
RAID Concepts (contd)
-
8/7/2019 Chapter1-Basic Storage Technology
23/48
Copyright 2006 by Patni 23 23
RAID Concepts (contd)
Parity
Data redundancy technique used in RAID.
Parity data is created using the logical operation
called XOR on the data elements.If any of the data elements is lost, it is recreated
from the parity element and vice versa.
As in mirroring there is no need to keep two copies
of data.The parity can be either distributed across the
multiple disks or be dedicated to a single disk.
-
8/7/2019 Chapter1-Basic Storage Technology
24/48
Copyright 2006 by Patni 24 24
RAID Levels
Combinations of mirroring, parity and striping results in
various raid levels.Commonly a op e RAID Levels
RAID 0 Striping (no parity)
RAID 1 Disk mirroring
RAID 0+1 Striping, each stripe then mirrored
RAID 2 Bit-level Striping, ECC Disk
RAID 3 Byte-level Striping, fixed parity
RAID 4 Block-level Striping, fixed parity RAID 5 Striping, distributed parity
RAID 6 is two parity over all drives, Handles twodisk failures.Handles twodisk failures.
-
8/7/2019 Chapter1-Basic Storage Technology
25/48
Copyright 2006 by Patni 25
25
RAID 0: Striping
Implements a striped disk array, the data is broken into
blocks each block written to separate disk drive
Not fault-tolerant not a true RAID
Lowercost & higher access rate
Applications in Image Editing & Video Production
Block7
Block5
Block3
Block1
Block8
Block6
Block4
Block2
Disk1 Disk2
-
8/7/2019 Chapter1-Basic Storage Technology
26/48
Copyright 2006 by Patni 26
26
RAID 1 : Mirroring
Consists exactly 2disk modules bound together as mirrored pair.
Controller must perform 2concurrent Reads or2duplicate
Writes, per mirrored pair.
Ifbothdisks fail, the RAID 1 mirrored pairbecomes inaccessible
Recommended Application-Accounting & Payroll.
Block4
Block3
Block2Block1
Block4
Block3
Block2Block1
Disk1 Disk2
-
8/7/2019 Chapter1-Basic Storage Technology
27/48
Copyright 2006 by Patni 27
27
RAID 0/1 : Striping & Mirroring
Even number of 4-16 disk modules
Half are data disks and the other half are disk mirrors
Uses block striping for performance & mirroring for redundancy
- so mirrored RAID 0 group
Block7
Block5
Block3
Block1
Block8
Block6
Block4
Block2
Block7
Block5
Block3
Block1
Block8
Block6
Block4
Block2
-
8/7/2019 Chapter1-Basic Storage Technology
28/48
Copyright 2006 by Patni 28
28
Uses bit level striping with dedicated Error Correction Code (ECC).
Use multiple drive dedicated ECC disks.
Need high number of drivers for ECC generation.
Block4
Block3
Block2
Block1
Block4a
Block3a
Block2a
Block1a
Block4b
Block3b
Block2b
Block1b
ECC
ECC
ECC
ECC
ECC
ECC
ECC
ECC
RAID 2 : Bit-level Striping, Fixed parity
-
8/7/2019 Chapter1-Basic Storage Technology
29/48
Copyright 2006 by Patni 29
29
RAID 3 : Byte-level Striping, Fixed parity
Uses byte level striping with dedicated parity
An additional drive dedicated to parity
Uses Error Correction Code to detect errors
Added parity slow down writes
Block4
Block3
Block2
Block1
Block4a
Block3a
Block2a
Block1a
Block4b
Block3b
Block2b
Block1b
parity
parity
parity
parity
-
8/7/2019 Chapter1-Basic Storage Technology
30/48
Copyright 2006 by Patni 30
30
RAID 4 : Block-level Striping,Fixed parity
Uses block level striping with dedicated parity
Multi user read, single user writes .
Uses Error Correction Code (ECC) to detect error.
Added parity slow down writes.
Block10
Block7
Block4
Block1
Block11
Block8
Block5
Block2
Block12
Block9
Block6
Block3
parity4
parity3
parity2
parity1
-
8/7/2019 Chapter1-Basic Storage Technology
31/48
Copyright 2006 by Patni 31
31
Uses block level striping with distributed parity
Data transferred to disk by independent read and write
operations
Combination of redundancy, cost effectiveness and storage
efficiency
Good for multitasking environment.
parity
Block1b
Block1a
Block1
Block2b
parity
Block2a
Block2
Block3b
Block3a
parity
Block3
Block4b
Block4a
Block4
parity
RAID 5 Striping and Distributed
Parity
-
8/7/2019 Chapter1-Basic Storage Technology
32/48
-
8/7/2019 Chapter1-Basic Storage Technology
33/48
Copyright 2006 by Patni 33
33
Solutions for Connectivity
Fibre Channel
iSCSI (Internet Small Computer System Interface)
FCIP (Fibre Channel Over Internet Protocol)
iFCP ( Internet Fibre Channel Protocol )
-
8/7/2019 Chapter1-Basic Storage Technology
34/48
Copyright 2006 by Patni 34
34
Introduction to Fibre Channel
High Speed: 2 Gbps, full duplex dedicated
connection.
Provides a general hardware transport vehicle for
Upper Level Protocols (e.g. SCSI, IP, etc)
Reduced congestion
Long Distance: Up to 10km.
Heterogeneous systems support
-
8/7/2019 Chapter1-Basic Storage Technology
35/48
Copyright 2006 by Patni 35
35
Fibre Channel Architecture
ATM
FC - ATM
IP
FC Link
Encapsulation
FC - LE
ULP (Upper Level Protocol) SCSI-3
SCSI - 3 Command
Set MappingFC-4
IPI - 3 Command
Set Mapping
(IPI-3 STD)
FC-3 Common Services
FC-
FC-1
FC-2Fibre Channel Physical
& Signaling Interface
( FC- PH, FC-PH2,
FC-PH3 )Physical Variant
Encode / Decode
Framing ProtocolFC - AL
8B/10B Encoding
Copper, Optical
FC - AL -2
-
8/7/2019 Chapter1-Basic Storage Technology
36/48
Copyright 2006 by Patni 36
36
The Fibre Channel Network
Topology
SCSI Target Node
SCSI Initiator Node
Link
IP Node
Name=1000.3210IP NodeName=1230.3210
Server
StorageSubsystem
SCSI CommandProtocol
-
8/7/2019 Chapter1-Basic Storage Technology
37/48
Copyright 2006 by Patni 37
37
Only 2devicesDirect Connect
Up to126devices
FC Hubs
Up to16 milliondevices
FC switches
Switche FabricArbitrate LoopPoint To Point
Fibre Channel Topologies
-
8/7/2019 Chapter1-Basic Storage Technology
38/48
Copyright 2006 by Patni 38
38
Fibre Channel Node and Port
Node Name=1000ABCD
Nodes
Node Name=6000..1254
N_Port
#1
N_Port
#2
N_Port#3
NL_Port#4
U_Port U_Port
NL_Port
#7
NL_Port
#8
Node
PortName=21008765
N_Port_ID = 123456Each Port hasseparate Transmit andReceive functions
Node: Any device or entity that can initiate and receive transmissionsin a fibre channel network.Eg. Storage device
Port: A fibre Channel port is an intelligent physical hardware thatconnects all components to the network.
Switch and hubs are not considered as nodes as they do not initiate
any transmissions.
A node can have multiple ports
-
8/7/2019 Chapter1-Basic Storage Technology
39/48
Copyright 2006 by Patni 3939
Fibre Channel Addressing
Each Node has a unique Node_Name ( 64-bit WWNN)assigned by the manufacturer.
As in LAN MAC address, WWN not used for transportation offrames across the network.
Each Node has one or more ports called N_Port (NL_Port)
Each N_Port has a 64-bit Port Name and a 24-bit port addressor N_Port_ID
The 24-bit address is assigned dynamically and optimizesframe routing.
64 bit Node_Name orWWN
Node Port orN_Port64 bit Port Name
24 bitN_Port_ID orport address
Multi-ported Node
-
8/7/2019 Chapter1-Basic Storage Technology
40/48
Copyright 2006 by Patni
40 40
Fibre Channel Port Types
Switch Ports
E _Port = Expansion Port
F _Port = Fabric Port
FL_Port = Fabric Loop Port
G _Port = Generic Port Can Operate as E, F or FL
Device Ports
N _Port = Direct Fabric Attached Device
NL_Port = Loop Attached Device
-
8/7/2019 Chapter1-Basic Storage Technology
41/48
Copyright 2006 by Patni
41 41
Port Visualization
N-Port Node
N-PortNode F-Port F-Port
E-Port
E-Port
FL-Port
G-Port
NL-Port
NL-Port
Switch 1
Switch 2
Fabric
Brocade
-
8/7/2019 Chapter1-Basic Storage Technology
42/48
Copyright 2006 by Patni
42 42
Classes of Service
High speed and Flexible architecture of Fibre Channel enablesit to carry out different applications.eg tape-backup, real-time
video etc.
Different applications have their own delivery requirements
like
bandwidth, connectivity etc.
Fibre Channel defines 5 classes of service that address
different delivery requirements
Class 1- Acknowledged Connection Oriented Service
Class 2- Acknowledged Connection-less Service
Class 3 - Unacknowledged Connection-less Service
Class 4 Fraction Bandwidth Connection Oriented ServiceClass 6 Multicast Service.
-
8/7/2019 Chapter1-Basic Storage Technology
43/48
43Patni Internal 43
Thanks
-
8/7/2019 Chapter1-Basic Storage Technology
44/48
Copyright 2006 by Patni
44 44
Class 1 Service
N_Port
# 1
N_Port
#2
N_Port
#3
N_Port
#4
Fabric
Connection-Oriented Class of Service. Allows full-bandwidth between a pair of nodes with a
confirmation of delivery and End To End Flow Control
In-Order delivery of frames due to dedicated connection.
Best suited for real-time applications like video processing.
Connections are established and removed at sequence boundaries.
-
8/7/2019 Chapter1-Basic Storage Technology
45/48
Copyright 2006 by Patni
45 45
Class 2 Service
N_Port
#2
N_Port
#1Frame 1:5
N_Port
#5
N_Port
#2
Frame 1:5
Frame 2:1
Frame 2:2
Connection-less service class and no bandwidth is reserved.
Frames are routed by the fabric using internal routing algorithms.
Do not guarantee In-Order delivery of frames, but there is
confirmation of delivery.
-
8/7/2019 Chapter1-Basic Storage Technology
46/48
Copyright 2006 by Patni
46 46
Class 3 Service
It is a connection-less and unacknowledged service.
Frame routing done internally by the fabric.
Also known as the datagram service, as it is the quickesttransmission without involving the acknowledgement.
If a frame cannot be processed then, it is discarded withoutnotification.
First frame of sequence
Data Frame
Last data frameFirst frame of sequence
Data Frame
Last data frame
Sequence
Initiator
Sequence
Recipient
Sequence
Initiator
Sequence
Recipient
-
8/7/2019 Chapter1-Basic Storage Technology
47/48
Copyright 2006 by Patni
47 47
Class 4 Service
This service is connection-oriented providing a virtual circuitbetween a pair of nodes.
Known as Fractional Bandwidth, as it assigns only a fraction ofthe total bandwidth and different QoS parameters for each
connection.
The QoS parameters for each services ensures that certain time-
critical applications always have bandwidth available.
Exists only as a standard due to its complexity.
-
8/7/2019 Chapter1-Basic Storage Technology
48/48
Copyright 2006 by Patni
48 48
Class 6 Service
Initiator
N_Port
Multicast
server
N_Port
N_port
N_Port
RecipientFabricConnectrequest
ACK
ACKs
from the
targets
Provides reliable multicast service with acknowledged delivery. It is only a variation from Class1 service in a way the responsesare processed by recipients of the multicast group.
A multicast server serves consolidates all acknowledgements,thereby returning only a single confirmation to the initiator.
Example, A video broadcast application, with a central videoserver and multiple video recipients.