2010 Storage Developer Conference. All Rights Reserved.
The Pieces to Managing FCoE
John CrandallBrocade
2010 Storage Developer Conference. All Rights Reserved.
Abstract
This presentation will identify and provide an overview to the many implementation pieces a client and FCoE switch and CNA vendor may need to consider for managing an FCoE environment including discovery and configuration. These pieces may include standards from the SNIA, DMTF, and T11. The presentation will attempt to identify the possible gaps in these standards and how they may be addressed and will also examine whether a client can hooks all these pieces together and if so how?
2
2010 Storage Developer Conference. All Rights Reserved.
Learning Objectives
What are the current and future management standards
How do you hook the pieces together What are the gaps and how might they be addressed
3
2010 Storage Developer Conference. All Rights Reserved.
Agenda
T11 SM-HBA IIFC-GS
SNIA SMI
DMTFFC HBA Diagnostics SVPC Virtual Networks
4
2010 Storage Developer Conference. All Rights Reserved.
SM-HBA-II
2010 Storage Developer Conference. All Rights Reserved.
Fabric/Nx_Port
VF_ID, in-order delivery, …
Terminology Translation
FabricFabric nameVFID, inorder,…VFID mask
0-n0-n
0-n
0-1FC Phy
Phy index, speed, topology, physical attributes
HBAHBA handle, manufacturer,model #, serial #, firmware version, ... # of Ports
FC PortPort index, Port name, Port address, Fabric name, ...# of Phys=1
0-1
0-n
0-n
0-1
1 0-n
1 1
1 1
0-n 1
Port statisticstraffic counters: frames, words
Protocol statisticsFC-4 TYPE, I/O counters: Input operations, output operations, control operations
Phy statisticstraffic counters: frames, wordslink error counters
Physical HBAPHBA handle, manufacturer, model #, serial #, # Ports
0-10-1Bus Address
variants for different standard buses
0-10-1
1
1-n
Bold borders: physical objects
VFID Mask,Port VFID
Fabric/Phy
Fabric/VN_Port
VF_ID, in-order delivery, …
VN_PortPort index, Port name, Port address, Fabric name, ...# of Phys=1
PN_PortPhy index, speed, topology, physical attributes
VFID Mask,Port VFID
Fabric/PN_Port
VN_Port statisticstraffic counters: frames, words
PN_Port statisticstraffic counters: frames, wordslink error counters
FC-4 statisticsFC-4 TYPE, I/O counters: Input operations, output operations, control operations
2010 Storage Developer Conference. All Rights Reserved.
What needs to be added
FCoE_LEPsVN_Port MAC address/VLAN ID and FCF-MAC
address Lossless Ethernet MACs Physical attributes, e.g., speed
FCoE ControllersENode MAC addressFC-BB_E VLAN mask
7
2010 Storage Developer Conference. All Rights Reserved.
PHYPhy index, speed, topology, physical attributes
0-n
0-1
Fabric/VN_Port
VF_ID, in-order delivery, …
Adding FCoE into the SM-HBA-2 Architecture
FabricFabric nameVFID, inorder,…VFID mask
0-n0-n
0-n
0-1
HBAHBA handle, manufacturer,model #, serial #, firmware version, ... # of Ports
PortPort index, Port name, Port address, Fabric name, ...# of Phys=1
0-1
0-n
0-n
0-1
1 0-n
1 1
1 1
0-n 1
Port statisticstraffic counters: frames, words
Protocol statisticsFC-4 TYPE, I/O counters: Input operations, output operations, control operations
PHY statisticstraffic counters: frames, wordslink error counters
Physical HBAPHBA handle, manufacturer, model #, serial #, # Ports
0-10-1Bus Address
variants for different standard buses
0-10-1
1
1-n
Bold borders: physical objects
VFID Mask,Port VFID
Fabric/PHY
PHYPhy index, speed, topology, physical attributes
2010 Storage Developer Conference. All Rights Reserved.
Fabric/VN_Port
VF_ID, in-order delivery, …
Adding FCoE into the SM-HBA-2 Architecture
FabricFabric nameVFID, inorder,…VFID mask
0-n0-n
0-n
0-1FCoE PHY (≡Lossless E’net MAC)
Phy index, speed, topology (?), physical attributes
HBAHBA handle, manufacturer,model #, serial #, firmware version, ... # of Ports
VN_Port Port index, Port name, Port address, Fabric name, ...# of Phys=1
0-1
0-n
0-n
0-1
1 0-n
1 1
1 1
0-n 1
Port statisticstraffic counters: frames, words
Protocol statisticsFC-4 TYPE, I/O counters: Input operations, output operations, control operations
PHY statisticstraffic counters: frames, wordslink error counters
Physical HBAPHBA handle, manufacturer, model #, serial #, # Ports
0-10-1Bus Address
variants for different standard buses
0-10-1
1
1-n
Bold borders: physical objects
VFID Mask, Port VFIDFabric/PHY
ENode ControllerEnode MAC address, FC-BB_E VLAN mask
1
0-n
Addresses*FCoE_LEP
*Addresses: VN_Port MAC,FCF_MAC,VLAN ID
FIP statisticstraffic counters: frames, wordslink error counters
1 1
2010 Storage Developer Conference. All Rights Reserved.
Additional details
From T11/09-530v1 (23 November 2009) by Bob Nixon (Emulex)
10
2010 Storage Developer Conference. All Rights Reserved.
FC-GS-7
2010 Storage Developer Conference. All Rights Reserved.
FC Switching Element and Physical Switch AttributesFC Switching
ElementObject
PhysicalSwitchObject
NameTypeDomain ID(s)Fabric NameLogical NameManagement Address(es)Physical Switch CorrelatableIdentifier
Correlatable IdentifierManagement Address(es)Vendor NameModel NameRelease CodeVendor Specific Info
2010 Storage Developer Conference. All Rights Reserved.
Vx Port and Px Port Attributes
Vx PortObject
Physical PortObject
NameTypeLogical NameAttached Port(s)Port StateZoning Enforcement StatusPhysical Port Correlatable ID
Transport Type (FC, FCoE, FCIP, …)Correlatable IDNameTx TypeModule TypePhysical Port NumberPort StatePort Speed CapabilitiesPort Operating SpeedPhysical Location
2010 Storage Developer Conference. All Rights Reserved.
Enhanced Fabric Configuration Server – Request Command CodesCode(hex)
Mnem. Description Attribute(s) inRequest CT_IU
Attribute(s) in Accept CT_IU
0100 GSL Get Switch List none List of Switch Names
0101 GSAG Get Switch Attribute Group Switch Name A group of attributes for the Switch
0102 GSPL Get Switch Port List Switch Name List of VxPorts
0110 GPSAG Get Physical Switch Attribute Group
Physical Switch Name
A group of attributes
0111 GPSPL Get Physical Switch Port List Physical Switch Name
List of PxPorts
0130 GVPAG Get Virtual Port Attribute Group
Physical Switch Name, List of Port Names
A group of attributes for the listed ports
2010 Storage Developer Conference. All Rights Reserved.
Enhanced Fabric Configuration Server –Request Command CodesCode(hex)
Mnem. Description Attribute(s) inRequest CT_IU
Attribute(s) in Accept CT_IU
0131 GAPL Get Attached Port List Port Name List of Attached Port Names
0140 GPPAG Get Physical Port Attribute Group
Physical SwitchName, List of Port Names
A group of attributes
2010 Storage Developer Conference. All Rights Reserved.
SNIA SMI
2010 Storage Developer Conference. All Rights Reserved.
C
MemberOfCollection
FCPortFCPort
FCPort ComputerSystem
Dedicated="Switch"
FCPort
PortType=”F”
SystemDevice
Switch(FC)ProtocolEndpoint
ProtocolType="FibreChannel"
DeviceSAPImplementation
HostedAccessPoint
ActiveConnection
ProtocolEndpoint
ProtocolType="FibreChannel"
DeviceSAPImplementation
FCPort
PortType=”N”
SystemDevice
ConnectivityCollection
LogicalPortGroup
MemberOfCollection
HostedCollection
Fabriic
AdminDomain
MemberOfCollection
AdminDomain
ContainedDomain
HostedAccessPoint
HostedCollection
Fabric Model
Current Fabric Model showing a Host and Switch Note
ConnectivityCollection collects PEs AdminDomain represents the Fabric and SAN (A System)
2010 Storage Developer Conference. All Rights Reserved.
C FCPortFCPort
FCPort ComputerSystem
Dedicated="Ethernet Switch"
EthernetPort SystemDevice
Switch(Ethernet)LANEndpoint
DeviceSAPImplementation
HostedAccessPoint
Host
LANEndpointDeviceSAP
Implementation
EthernetPort
ConnectivityCollection
AdminDomain
HostedCollection
HostedAccessPoint
MemberOfCollection
FCoE Additions
Mirror image of FC Fabric Ethernet Switch
Dedicated = Ethernet Switch
Host ConnectivityCollection collects
LANEndpoints AdminDomain represents the
Ethernet Cloud
Avoiding Ethernet Topology Work Items
Ethernet Switch Durable Name Cloud Durable Name
2010 Storage Developer Conference. All Rights Reserved.
ProtocolEndpoint
ProtocolType="FibreChannel"
FCPort
PortType=”N”
FCPort
PortType=”N”
SystemDevice
ConnectivityCollection
FC Node
LogicalPortGroup
MemberOfCollection
HostedCollection
Fabriic
AdminDomain
MemberOfCollection
Host
SAN
AdminDomain
ContainedDomain
HostedAccessPoint
HostedCollection
LANEndpoint
DeviceSAPImplementation
EthernetPort
ConnectivityCollection
AdminDomain
MemberOfCollection
HostedCollection
FCPortFCPort
FCPort ComputerSystem
Dedicated="FC Switch"
FCPort
PortType=”F”PortDiscriminatorr=”Internal”
SystemDevice
Switch(FC)
ProtocolEndpoint
ProtocolType="FibreChannel"
DeviceSAPImplementation
HostedAccessPoint
HostedAccessPoint
ActiveConnection
MemberOfCollection
FCPortFCPort
FCPort ComputerSystem
Dedicated="Ethernet Switch"
EthernetPortSystemDevice
Switch(Ethernet)
LANEndpoint
DeviceSAPImplementation
HostedAccessPoint
HostedDependency
ComputerSystem
Dedicated=""Flavor=”Physical”
HostedDependency
HostedDependency
FCPort
PortType=”N”PortDiscriminator=“FCoE”
DeviceSAPImplementation
FCPort
PortType=”F”
FCPort
PortType=”F”
FCPort
PortType=”F”
SystemDevice
FCPort
PortType=”F”
FCPort
PortType=”F”
ProtocolEndpoint
ProtocolType="FibreChannel"
DeviceSAPImplementation
CNA
All together
HostedDependency relates FC to Ethernet Probably need to show binding (FCoE) relationship between
PEs Partitioning Model connects the two systems (physical
system not shown in this diagram) Many to Many relationship between the Ethernet Switch
EthernetPort and the FC Switch FCPort
2010 Storage Developer Conference. All Rights Reserved.
What is missing
VLANs ACLs
20
2010 Storage Developer Conference. All Rights Reserved.
DMTF Diag Profiles
2010 Storage Developer Conference. All Rights Reserved.
Ethernet NIC
Define a common industry standard diagnostics interface for clients toDiscover and execute diagnostic testsConfigure and control diagnostic test executionMonitor diagnostic test progressView and manage diagnostic test execution results
FC HBA (DSP1104) Ethernet NIC (DSP1006)
22
2010 Storage Developer Conference. All Rights Reserved.
ManagedElement isPortControllerEthernetPort
Diagnose NIC problems Pre-boot vs online tests Subcomponent tests
23
2010 Storage Developer Conference. All Rights Reserved.
SVPC Virtual Switch
2010 Storage Developer Conference. All Rights Reserved.
25
Profiles almost done
DSP 1097 – Virtual Ethernet Switch Profile Specializes the autonomous Virtual System Profile. This
profile defines the minimum top-level object model needed to define a virtualization system’s internal Ethernet switch.
DSP 1050 – Ethernet Port Resource Virtualization Profile Specializes the abstract Resource Allocation Profile and
the Allocation Capabilities Profile to specify the allocation and management of a host computers Ethernet network access, in support of virtual computer systems ability to access external and internal network.
2010 Storage Developer Conference. All Rights Reserved.
But there is more …
IEEE Port Profile WorkVEBs –Virtual Ethernet BridgeVEPAs – Virtual Ethernet Port Aggregator
26
2010 Storage Developer Conference. All Rights Reserved.
Questions?