extending data centers for application and server virtualization-apac-april 2010-v3 · optics...
TRANSCRIPT
Extending Data Centers for Application and Server Virtualization
© 2006 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 1
Yousuf Hasan – Manager, Data Center Systems Marketing
Agenda
� DCI Concepts and Business Drivers
� DCI components
� LAN extensions
� Current offerings
� Dark Fiber
� MPLS
� Future offerings
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 2
� A-VPLS
� OTV
� Routing optimization
� Storage extensions
� Application mobility
� DCI joint vendor validation with VMware and storage partners
� DCI Systems level validation
Data Center Interconnect (DCI) Concept
� Data Centers are extending beyond traditional boundaries:
� Distributed Applications: extending Operating System, File System clusters, Database clusters
� Business Continuity: Workload mobility, Time to deployment and operational reasons (maintenance, power/heat/space limitations), Disaster Prevention, Data Center Consolidation or expansion .
LAN Extension
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 3
� Legacy devices/applications with embedded IP addressing
� Cloud Computing Business Models
� Virtualization applications are driving DCIacross Pods and Data Centers
Use Case Business Driver IT Solutions
Business Continuity
Business Resource
Optimization
� Data Center Maintenance/migration/consolidation
� Disaster Avoidance (hurricanes, floods, etc)
� Disaster Recovery
� Workload Mobility
� Application High Availability
� VMware VMotion
� VMware SRM
� Geo clusters
DCI Business DriversMarket Size/Growth – Server Virtualization
� Increase in Virtual Machine mobility is a major driver for DCI. Virtual Machine servers are more than 50% of the new servers by end of 2009
� According to recent research from IDC, the virtualization services
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 4
virtualization services market will grow from $5.5 billion in 2006 to $11.7 billion in 2011
� Virtualization will be the highest-impact trend changing infrastructure and operations…. Gartner, IncSource: Michelle Bailey, VP, IDC – September 2009 – VMWare Conference VM5401
VMotion across Data Centers
VMware
Data Center 1 Data Center 2
Data Center
Interconnect
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 5
VMwarevCenter
vSphere
NexusNexus1000V1000V
vSphere
NexusNexus1000V1000V
Data Center Interconnect (DCI)General Requirements
DCI Purpose
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 6
DCIAttributes
Purpose
LAN
Extensions
Extend same VLAN across Data Centers, to virtualizeservers and applications
Storage
Extensions
Providing applications access to storage locally, as well as remotely with desirable storage attributes
Routing
Optimization
Routing users to the data center where the application resides while keeping symmetrical routing in consideration for IP services (e.g. Firewall)
Application
Mobility
Enablers to extend applications across data centers (e.g. VMware VMotion)
Agenda
� DCI Concepts and Business Drivers
� DCI components
� LAN extensions
� Current offerings
� Dark Fiber
� MPLS
� Future offerings
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 7
� A-VPLS
� OTV
� Routing optimization
� Storage extensions
� Application mobility
� DCI joint vendor validation with VMware and storage partners
� DCI Systems level validation
DCI LAN Extensions
Challenges
� Spanning Tree isolation between data center sites
� End-to-end loop prevention
� Storm and flood control between data center sites
Data Center 1
Data Center 2
Access
Agg/DCI
E
S
X
E
S
X
Core
� Efficient use of WAN bandwidth
LAN Extension
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 8
� Efficient use of WAN bandwidth
� Scale multiple sites, VLANs, MAC addresses
� DCI layer redundancy, HA
Additional Requirements
� WAN link encryption (IPSec, 802.1AE)
� H-QoS (sub-rate SP servcie, EVPL service)
DCI LAN ExtensionsCisco options today
C6K�VPLS
2 Data Center Sites Multi-Site Data Centers
N7K�vPC
Dark Fiber
C6K�VSS
ASR-1KC6K
N7K�vPC
C6K�VSS
MPLS
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 9
� A-VPLS*
�VPLSoGRE
�A-VPLSoGRE*
OTV
�EoMPLS
�EoMPLSoGRE
�EoMPLS
�EoMPLSoGRE
N7K�OTV*
N7K�OTV*
MPLS
* Available Q2 CY10
Agenda
� DCI Concepts and Business Drivers
� DCI components
� LAN extensions
� Current offerings
� Dark Fiber
� MPLS
� Future offerings
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 10
� A-VPLS
� OTV
� Routing optimization
� Storage extensions
� Application mobility
� DCI joint vendor validation with VMware and storage partners
� DCI Systems level validation
LAN Extension (vPC/VSS)
� 6500 with Virtual Switching System cluster
� Nexus 7000 with Virtual Port-Channels
� Extend VSS or vPC across dark fiber (upto 100 kms)
� VSS and vPC pairs can inter-operate using LACP
� Can consume lambda or fiber strands quickly
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 11
� Can consume lambda or fiber strands quickly
� Data plane rate limiting in L2 still needs protection
� STP domains are not isolated unless we BPDU filter at all vPC/VSS aggregation switches
LAN Extension VSS/vPC2 Sites
SiSiSiSinPE nPE
FIBER/ DWDM/CWDMCORE
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 12
VSL
VSLSiSiSiSi
nPE
Aggregation
Access
VSLSiSi
MEC
Aggregation
SiSi
MEC
Access
SiSiSiSi
DC1 DC2
MEC
DC2
LAN Extension VSS, vPCMulti Site
VSL, vPCSwitches use separate
Lambda to
Interconnect
Switches use separate
Lambda to
Interconnect
DWDM CORE
SR SR
Physical View
DCI DCI
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 13
VSLAggregation
VSL VSLSiSi
MEC MECMEC
Access
SiSi
SR
Optics
SR
Optics
SiSi
DC1 DC4DC2 DC3
SiSi SiSiSiSi
MEC
vPC
MEC
SR
Optics SR
Optics
VSS, vPC Core
Switches reside
In 2 primary DC’s
λ λ λ λ λ λ
Logical View
LAN Extension VSS, vPCMulti Site
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 14
Aggregation
Access
MEC MEC
MEC Optics
DC1 DC2 DC3 DC4
MEC
Spanning Tree Isolation:
DCIVSL
MEC
MEC MEC
MEC
BPDU Filtering enabled to Filter any BPDUs
(config-if)spanning-tree bpdufilter enable
With MEC (VSS/vPC) at the DCI Layer
Appears as a Single switch and removes the
need for Spanning Tree
Spanning Tree Considerations
With MEC (VSS or vPC) there is no Single
Link or Device Failure will cause Data Center
Isolation
High Availability:
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 15
Aggregation
Access
VSL
MEC MEC Isolation
With VSS + MEC, vPC, We Achieve Physical
Device and Network Redundancy yet logically
Bundling the device as 1 Destination for the
previous layer in the Architecture and thus
removing a topology which will create loops
Avoiding Loops:
With MEC (VSS/vPC) there is no need for
Spanning tree, no ports are Blocked. Packets
Are Load-balanced Across the Etherchannel
Links achieving efficient Link Utilization
Multipath:
SiSi SiSi
STP STP STP
GR3 Ring OpticalProtection >50ms
LAN extension with ONS Xponder
Run 1+1 between Cat6K / N7K & XPE(s)
Provide Client Protection
Run 1+1 between Cat6K / N7K & XPE(s)
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 16
STPDomain #1
STPDomain #2
STPDomain #3
The ONS 15454 10GE-XPE in DCI Architecture� Provide STP Isolation between N number of Data Centers (block STP BPDUs on
ingress port)� Reduces Wavelength Explosion (a single 10G λ needed to interconnect N number of
Data Centers)� Remove the possibility of the VSS Star Topology. The 10 Gbps Optical Ring is shared� Intelligent multicast delivery through igmp snooping� Broadcast control� Less than 50ms failure restoration (GR3 protection)� 1+1 port redundancy
Agenda
� DCI Concepts and Business Drivers
� DCI components
� LAN extensions
� Current offerings
� Dark Fiber
� MPLS
� Future offerings
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 17
� A-VPLS
� OTV
� Routing optimization
� Storage extensions
� Application mobility
� DCI joint vendor validation with VMware and storage partners
� DCI Systems level validation
DCIAccess
Server FarmServer Farm
ServerFarmServerFarm
Agg
DC Core
Layer 3 CoreIntranet
DCI
MPLS Core
Active PW
Active PW
LAN ExtensionMPLS options: EoMPLS for 2 sites
� Port mode EoMPLS as LAN extension
between data centers
� Port mode requires BPDU filtering to
isolate spanning tree
� 6500 as DCI platform, requires use of
EEM to detect remote port shutdown
� Several encryption options available
SIP based EoMPLS model:
Physical View
Logical View
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 18
DCI Platform mode Encryption Advantages and design considerations
6500 Port mode 802.1AE from Agg-
Agg (N7K in Agg)
� Control plane can be tunneled across
� Load balancing between N-PEs
� Requires VSS based 6500 pair or vPC based
N7K pair in aggregationASR-1K Port mode 802.1AE from Agg-
Agg (N7K in Agg)
SIP based EoMPLS model:� does not allow end-end port channel
� allows only one active pseudowire. The other pseudowire
will be standby and will be activated via EEM
LAN ExtensionEoMPLSoGRE option: 2 sites
� EoMPLSoGRE as LAN extension
between data centers
� Port mode requires BPDU filtering to
isolate spanning tree
� Several encryption options availableDCI
Access
Server
Farm
Server
FarmServer
Farm
Server
Farm
Agg
DC Core
Layer 3 CoreIntranet
DCI
MPLS Core
IP Core
GRE Tunnel
GRE Tunnel
Active PW
Active PW
Physical View
Logical View
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 19
DCIPlatform
mode Encryption Advantages and designconsiderations
ASR-1K Port
mode
� 802.1AE from Agg-Agg (N7K in
Agg)
� IPSec from DCI-DCI layer (6500 in
Agg, N7k in Agg)
� Control plane can be tunneled across
� Load balancing between N-PEs
� Requires VSS based 6500 pair or
vPC based N7K pair in aggregation
SIP based EoMPLS model:� does not allow end-end port channel
� allows only one active pseudowire. The other pseudowire
will be standby and will be activated via EEM
LAN ExtensionMPLS options: VPLS for 2 or more sites
� VPLS as LAN extension
between data centers using
SIP-400 and SIP-600 on 6500
� 6500 as DCI platform, requires
use of EEM to provide N-PE
redundancy
� N-PE primary carries odd
VLANs, N-PE backup carries
even VLANs to utilize DCI
DC Core
Layer 3 CoreIntranet
DC Core
Layer 3 CoreIntranet
MPLS Core
Active PWPrimary N-PE
Primary N-PE
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 20
even VLANs to utilize DCI
bandwidth
Access
Server FarmServer Farm
Agg
Access
Server FarmServer Farm
Agg
DC Core
Access
Server FarmServer Farm
Agg
Layer 3 CoreIntranet
MPLS Core
Backup N-PE
Backup N-PE
Backup N-PE
Primary N-PE
LAN ExtensionMPLSoGRE options: VPLSoGRE for 2 or more sites
� For DCI multipoint connectivity over
IP core
� VPLSoGRE as LAN extension
between data centers using SIP-400
on 6500
� Loopback cable required for single
PE PECE CE
VPLS Multipoint Services
VFI VFI
VFI
IP Core
GRE
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 21
� Loopback cable required for single
box solution with 6500 and IPSec
encryption
� 6500 as DCI platform, requires use
of EEM to provide N-PE redundancy
� N-PE primary carries odd VLANs,
N-PE backup carries even VLANs to
utilize DCI bandwidth
CE
VFI
Agenda
� DCI Concepts and Business Drivers
� DCI components
� LAN extensions
� Current offerings
� Dark Fiber
� MPLS
� Future offerings
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 22
� A-VPLS
� OTV
� Routing optimization
� Storage extensions
� Application mobility
� DCI joint vendor validation with VMware and storage partners
� DCI Systems level validation
Easy Configuration
LAN ExtensionA-VPLS (Advanced VPLS)
Easy-to-use CLI for VLAN extn.Reduced Complexity!
DC DCSP /
Dark Fiber
Internet
SP service
L2/L3/L4 flow based Balancing�DC Edge to Aggregation
12.2(33)SXI4Enhanced
VPLS
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 23
Simplified Redundancy
Efficient BandwithUtilization
DCIA-VPLS
�DC Edge to Aggregation
�DC Edge to WAN
�WAN Core
�VSS Single Chassis Redundancy�Fast Sub-second Convergence
Ethernet LAN extension over MPLS or IP “Any flow Any Link” Load-balancingMultipoint loop-free connectivity
VPLS based LAN Extension Problem Statement
MPLS
IP
DC-2
VLAN10,20
DC-1
EEM
STP
EEM
STP
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 24
Challenges with MPLS-Based VLAN extensions
1. Complex Edge Redundancy (EEM scripts)
2. Sub-optimal Bandwidth Utilization (one N-PE active per VLAN group)
3. VPLS Configuration complexity
A-VPLS based LAN Extension Simplified Redundancy
MPLS/IP
VSS-VPLS on SIP400
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 25
VSSVSS
Benefits
� Single VSS Box for DC Edge Redundancy
� Native VSS/MEC Failover – No scripts!!
� Sub-second Failover - No STP or EEM Dependency
A-VPLS based LAN Extension Efficient Load-Balancing
MPLS/IP
MECMEC
L2/L3/L4 Load-Balancing
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 26
• “Virtual Ethernet” Port-Channel for Bundling MPLS/IP links
• End-to-end L2/L3/L4 Load-Balancing on Virtual Ethernet Ports, MEC Port and any
source-destination pair over WAN
• VSS-Port Channel over WAN – No EEM Blocked Links
• VSS-MEC towards DC Aggregation – No STP Blocked Links
VSS VSSVirtual Ethernet
A-VPLS based LAN Extension Easy Configuration
MPLS
IP
DC-2
Vlan1,1000
DC-1
EEM
STP
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 27
Traditional VPLS Configuration
Interface Vlan 1
xconnect vfi …
……..
Interface vlan 1000
xconnect vfi …
A-VPLS Configuration
interface virtual-ethernet 1
switchport
switchport trunk allowed vlan 1,1000
Agenda
� DCI Concepts and Business Drivers
� DCI components
� LAN extensions
� Current offerings
� Dark Fiber
� MPLS
� Future offerings
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 28
� A-VPLS
� OTV
� Routing optimization
� Storage extensions
� Application mobility
� DCI joint vendor validation with VMware and storage partners
� DCI Systems level validation
LAN ExtensionOverlay Transport Virtualization (OTV)
� Ethernet traffic between sites is encapsulated in IP: “MAC in IP”
� Dynamic encapsulation based on MAC routing table
� No Pseudo-Wire or Tunnel state maintained
IP A ���� IP B MAC1 � MAC2
Q2 CY10N7K
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 29
Communication between
MAC1 (site 1) and MAC2 (site 2)Server 1
MAC 1
Server 2
MAC 2
OTV OTV
MAC IF
MAC1 Eth1
MAC2 IP B
MAC3 IP B
IP A IP B
Encap Decap
MAC1 � MAC2 IP A ���� IP B MAC1 � MAC2 MAC1 � MAC2
MAC TABLE
VLAN MAC IF
100 MAC 1 Eth 2
100 MAC 2 Eth 1
100 MAC 3 IP B
100 MAC 4 IP B
LAN Extension: OTV Data Planeunicast
OTV Inter-Site Traffic
MAC Table contains
MAC addresses reachable through
IP addresses
Layer 2
Lookup
1
MAC TABLE
VLAN MAC IF
100 MAC 1 IP A
100 MAC 2 IP A
100 MAC 3 Eth 3
100 MAC 4 Eth 4
Layer 2
Lookup
5
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 30
Eth 4
Eth 3
100 MAC 4 IP B
MAC 2
MAC 1 Core
MAC 4
MAC 3
OTVOTV
IP A IP B
West East
L2 L3 L3 L2
IP A � IP BMAC 1 � MAC 3
OTVOTV
Encap
2
� No Pseudo-Wire state is maintained.
� The encapsulation is done based on a destination
lookup, rather than based on a circuit lookup.
3 Decap
4 MAC 1 � MAC 3
6MAC 1 � MAC 3IP A � IP BMAC 1 � MAC 3
100 MAC 4 Eth 4
Eth 1
Eth 2
MAC 1 � MAC 3
OTV as LAN ExtensionBuilding MAC tables
� OTV uses a protocol to proactively advertise MAC reachability (control-plane learning). We will refer to this protocol as the “overlay Routing Protocol” (oRP).
� oRP runs in the background once OTV has been configured.
� No configuration is required by the user for oRP to operate.
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 31
Cor
e
IP A IP B
IP C
West East
South
oRP
OTV as LAN ExtensionCore requirements
� The core needs to be able to delivery IP packets.
� No upgrades to the core are required to deploy OTV
� OTV is transport agnostic. OTV could be supported by any transport in the core: IP, MPLS, etc.
� OTV can leverage the benefits of an IP core: native IP
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 32
� OTV can leverage the benefits of an IP core: native IP multicast
� Multicast support in the core is strongly recommended to fully benefit from the OTV advantages. However multicast is not a must.
OTV as LAN ExtensionSTP BPDU Handling
� When STP is configured at a site, an Edge Device will send and receive BPDUs on the internal interfaces.
� An OTV Edge Device will not originate or forward BPDUs on the overlay network.
� An OTV Edge Device can become (but it is not required to) a root of one or more spanning trees within the site.
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 33
one or more spanning trees within the site.
� An OTV Edge Device will take the typical action when receiving Topology Change Notification (TCNs) messages.
OTVOTV
Core
The BPDUsstop here
OTV as LAN ExtensionUnknown Unicast Packet Handling
� Flooding of unknown unicast over the overlay is not required and is therefore suppressed.
� Any unknown unicasts that reach the OTV edge device will not be forwarded onto the overlay.
� The assumption here is that the end-points connected to the network are not silent or uni-directional.
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 34
are not silent or uni-directional.
� MAC addresses for uni-diretional host are learnt and advertised by snooping the host’s ARP reply
OTVOTV
Core
No MAC 3 in theMAC Table
MAC 1 � MAC 3
MAC TABLE
VLAN MAC IF
100 MAC 1 Eth1
100 MAC 2 IP B
OTV as LAN ExtensionProxy ARP
� The Edge Devices handle all ARP information for hosts across sites
� OTV provides the ability to implement Proxy ARP at the Edge Devices.
� With Proxy ARP the Edge devices may reply to ARPs on behalf of hosts on remote sites.
� With Proxy ARP the ARP broadcasts traffic crossing the core can be
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 35
� With Proxy ARP the ARP broadcasts traffic crossing the core can be significantly reduced.
Agenda
� DCI Concepts and Business Drivers
� DCI components
� LAN extensions
� Current offerings
� Dark Fiber
� MPLS
� Future offerings
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 36
� A-VPLS
� OTV
� Routing optimization
� Storage extensions
� Application mobility
� DCI joint vendor validation with VMware and storage partners
� DCI Systems level validation
Routing OptimizationProblem Statement
� VMotion moves VM between data centers while retaining the same
IP and MAC address
� Ingress traffic should be routed to new VM location
� Egress traffic must be symmetrical to maintain stateful services
(firewall)
Options Ingress Traffic Egress Traffic
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 37
DNS � DNS based solution using
GSS/ACE
� HSRP Localization/Filtering
� Src-NAT on the ACE
Routing � Route Health Injection
WANData Center 1 Data Center 2
HSRPActive
HSRPStandby
HSRPActive
HSRPStandby
Cisco
GSS
Cluster
ACE1ACE2
DCI Routing OptimizationDNS based
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 38
Virtual Machine
sql-server.jsmp.cisco.com
Access
Agg
Access
Agg
ESX ESXVirtual Machine
sql-server.jsmp.cisco.com
VMwarevCenter Jointly validated architecture with Cisco, NetApp and VMWare
http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9402/white_paper_c11-591960.pdf
Agenda
� DCI Concepts and Business Drivers
� DCI components
� LAN extensions
� Current offerings
� Dark Fiber
� MPLS
� Future offerings
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 39
� A-VPLS
� OTV
� Routing optimization
� Storage extensions
� Application mobility
� DCI joint vendor validation with VMware and storage partners
� DCI Systems level validation
Storage ExtensionsMultiple approaches
Storage Architectures
Shared Storage Storage VMotion and Vmotion
Intelligent Data Management for VMotion
Stretched or Clustered Storage for VMotion
Description � Access remote
storage across
SAN
� move storage
between
datastores at
different sites.
VMotion
immediately
� At each site arrays
present identical storage
to hosts. Single
canonical copy of data at
origin.
� Data cached on non-
� Arrays appear as
single logical entity.
� Data moves and
copies coordinated via
management plane.
VM move has implications on accessing storage for application data.
There are 4 possibilities.
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 40
immediately
follows.
� Data cached on non-
origin arrays.
� VM granular.
Performance over Distance
Long distances
result in high latency and poor performance.
performance issues during initial transfer.
Supports extremely long
distances
Solutions under
development by storage
vendors
StorageRequirements
Low. No specialized
storage required.
Moderate. Storage systems
must be present at
all sites.
Moderate. Storage systems must be
present at all sites.
Caching configured at all
sites (zero size until filled).
High. Storage systems must be
present at all sites.
Clustering may add
complexity, which should
be addressed by storage
system
DCI Storage ExtensionsOption 1a - Shared Storage
Cisco MDS 9513
ESX ESXESXESX
Cisco MDS
9134
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 41
Stretched
Network
<400KM
•FibreChannel (same metro area, 100 kms)
•FCIP (extend storage over longer distances)
•iSCSI
•NFS (NAS)
VM accesses storage across data centers. Cisco MDS helps extend SAN
Pros: No storage movement
Cons: High latency and performance issues over longer distances
Cisco MDS 9513
ESX ESXESXESXESX ESXESXESX
Cisco MDS
9134
DCI Storage ExtensionsOption 1b - Storage VMotion
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 42
Stretched
Network
<200KM
Stretched
Network
<400KMStorage for VM
Move storage across data centers. Cisco MDS helps extend SAN
Pros: Local Storage for the VM.
Cons: Storage transfer time
•FibreChannel (same metro area, 100 kms)
•FCIP (extend storage over longer distances)
•iSCSI
•NFS (NAS)
DCI Storage ExtensionsOption 2 – Intelligent Data Management
ESX ESXESXESXESX ESXESXESX
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 43
VMFS-Site A
NFS-Site A
VMFS-Site B
NFS – Site BStretched
Network
<200KM
Stretched
Network
<400KM
DCI & NFS
Cached storage provides a local data origin and remote cache. Tested with NetApp
Flexcache
Pros: Local Storage for the VM without moving storage.
Cons: applications with too many writes may degrade cache performance
ESX ESXESXESXESX ESXESXESX
DCI Storage Extensions Option 3 – Stretched or Clustered
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 44
VMFS-Active-Active
Stretched
Network
<400KM
Single
Logical
Array
One Storage Array across data centers. In development by storage vendors
Pros: Local Storage for the VM without moving storage.
Cons: distance limitations across data centers. Typically writes are synchronized
Agenda
� DCI Concepts and Business Drivers
� DCI components
� LAN extensions
� Current offerings
� Dark Fiber
� MPLS
� Future offerings
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 45
� A-VPLS
� OTV
� Routing optimization
� Storage extensions
� Application mobility
� DCI joint vendor validation with VMware and storage partners
� DCI Systems level validation
DCI Application MobilityVMware VMotion Requirements
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 46
Network Requirements Storage Requirements
≥ 622 Mbps of bandwidth and ≤ 5 msec
latency between ESX Servers
The virtual disks of all the VMs are on
VMFS/NFS data stores
VMs have access to the same IP subnet
and broadcast domain on source and
destination ESX Servers
The VMFS/NFS data store read/write for
source and destination
Agenda
� DCI Concepts and Business Drivers
� DCI components
� LAN extensions
� Current offerings
� Dark Fiber
� MPLS
� Future offerings
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 47
� A-VPLS
� OTV
� Routing optimization
� Storage extensions
� Application mobility
� DCI joint vendor validation with VMware and storage partners
� DCI Systems level validation
VM Mobility joint testing with VMware, NetAppVMotion with Intelligent data management
� OTV as DCI mechanism
� Vmotion across data centers 400
kms apart
� NetApp flexcache Storage &
Vmotion with intelligent data
management
� SQL database as VM with e-
63.736372.22
85.45
123.07
156.12
0
20
40
60
80
100
120
140
160
180
0.25 ms RTT 1 ms RTT 2 ms RTT 3 ms RTT 4 ms RTT
Time
Time in
Seconds
Time taken to move VM inter-pod and inter-data center
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 48
commerce application
� Zero downtime while moving VM
across data centers
� Whitepaper available athttp://www.cisco.com/en/US/prod/collateral/
switches/ps9441/ps9402/white_paper
_c11-591960.pdf
0.25 ms RTT
(0 km)
1 ms RTT
(100 km)
2 ms RTT
(200 km)
3 ms RTT
(300 km)
4 ms RTT
(400 km)
Latency/Distance
11840
11803 11800
11609
11567
11400
11450
11500
11550
11600
11650
11700
11750
11800
11850
11900
0.25 ms RTT
(0 km)
1 ms RTT
(100 km)
2 ms RTT
(200 km)
3 ms RTT
(300 km)
4ms RTT
(400 km)Latency/Distance
OPM Less than 3% application
performance degradation
Application performance impact because of VMotion
Agenda
� DCI Concepts and Business Drivers
� DCI components
� LAN extensions
� Current offerings
� Dark Fiber
� MPLS
� Future offerings
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 49
� A-VPLS
� OTV
� Routing optimization
� Storage extensions
� Application mobility
� DCI joint vendor validation with VMware and storage partners
� DCI Systems level validation
Systems Architecture and Strategy UnitEnabling execution…
Application
Networking &
Switching TG
Application
Networking &
Switching TG
Application
Networking &
Switching TG
Data Center
Switching TG
Data Center
Switching TG
Data Center
Switching TG
PartnerIntegrationIntegration
ImplementationImplementation
Service
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 50
Systems Architecture and Strategy Unit
Ethernet
Switching TG
Ethernet
Switching TG
Ethernet
Switching TG Cloud Services
& Switching TG
Cloud Services
& Switching TG
Service
ProviderIntegrationIntegration
ImplementationImplementation
EnterpriseIntegrationIntegration
ImplementationImplementation
Systems Architecture and Strategy UnitProgram overview…
MissionProvide scalable flexible Data Center and Borderless solutions, which focus on real-world challenges, provide dramatic differentiation and result in significant reduction in implementation/integration.
Scope
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 51
Data Center InterconnectCisco Products & 3rd party products
Deliverables Design and Implementation Guide (DIG)Master Verification Publication (MVP)Transfer of Information (TOI)Focused Launch and Marketing Campaign
Systems Architecture and Strategy UnitProcess and deliverables…
Architecture design
development
Design verification
& integration
Test plan
development
Information
collection
Process
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 52
development
Systems
Architecture
Specification
(SAS)
& integration
Design &
Implementation
Guide
(DIG)
development
Master
Verification
Plan
(MVP)
collection
Systems
Requirements
Documentation
(SRD)
Deliverables
Phase 1 Phase 2 Phase 3 Phase 4
DCI System ValidationVSS/vPC over Dark Fiber, Xponder� VSS or vPC over Dark Fiber as LAN Extension
� Data Centers in same metro area
� VSS-VSS, VSS-vPC and vPC-vPC interoperability with 2-site and 3-sites
� Design Guide available at http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/ns949/ns304/ns975/data_center_interconnect_design_guide.pdf
� Xponder update to Design Guide available in early May 2010
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 53
VSLAggregation
VSL VSLSiSi
VSL, vPC
MEC MECMEC
Access
SiSi
Switches use separateLambda to
Interconnect
Switches use separateLambda to
Interconnect
DWDM CORE
SROptics
SROptics
SiSi
DC1 DC4DC2 DC3
SiSi SiSiSiSi
DCI
MEC
VSL
VSLSiSiSiSi
VSLSiSi
MEC
DCI
Aggregation
SiSi
FIBER/ DWDM/CWDMCORE
MEC
Access
SiSiSiSi
DC1 DC2
ECMP
DCI Systems ValidationMPLS based validation
In progress….Design Guide available in early May 2010
� Aggregation and DCI based failure scenarios and convergence guidance� DCI platform failures (link to core, link to aggregation layer, supervisor, line card OIR).
� DCI layer switch failure
� Aggregation platform failure (link to DCI layer, supervisor, line card OIR)
� Aggregation layer switch failure
� MPLS brownout failure
� VLAN and MAC address scalability guidance
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 54
� VMotion across Data Centers in above cases
DCIAccess
Server FarmServer Farm
ServerFarmServerFarm
Agg
DC Core
Layer 3 CoreIntranet
DCI
IP/MPLSCore
Active PW
Active PW
DC Core
Access
Server FarmServer Farm
Agg
Layer 3 CoreIntranet
DC Core
Access
Server FarmServer Farm
Agg
Layer 3 CoreIntranet
DC Core
Access
Server FarmServer Farm
Agg
Layer 3 CoreIntranet
MPLS Core
Active PW
Primary N-PE
Backup N-PE
Use Case IP/MPLS DCIPlatform
(2-site) (3-site) Encryption VLANsextended(unicast, multicast)
Comments(3k-5k IGP routes per
Data Center)
EoMPLS MPLS ASR-1K Port
mode802.1AE 1200 N7k in Aggregation
layer
MPLS 6500 Port
mode802.1AE 1200 N7k in Aggregation
layer
EoMPLS
oGRE
IP ASR-1K Port
mode802.1AE 1200 N7k in Aggregation
layer
DCI Systems ValidationMPLS based validation
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 55
oGRE layer
IP ASR-1K Port
modeIPSec 1200 Hybrid (N7k, 6500) in
Aggregation layer
VPLS MPLS 6500
SIP-600� � none 300 (no Q-
in-Q)
6500 in Agg, EEM
N7K in Agg, EEM
A-VPLS MPLS 6500
SIP-400� � none 300 (no Q-
in-Q)
6500 in Agg, N7K in
Agg
Interop with VPLS
DCI Systems validation OTV based validation
� Design Guide to be available in July 2010
� OTV as LAN Extension
� Routing Optimizations for Applications across Data Centers/Cloud
� Show business continuity in a Data Center environment with Virtual Machine Mobility (VMotion) with popular business applications such as SQL
� Services: H-QoS, Encryption
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 56
Summary
� DCI requires considerations for LAN extensions, SAN Extensions, Routing Optimization and Application mobility
� LAN extensions and SAN extensions provide different flavors
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 57
flavors
� For more information, please visit http://www.cisco.com/en/US/netsol/ns975/index.html
� Questions: [email protected]
© 2010 Cisco Systems, Inc. All rights reserved. All specifications subject to change without noticePresentation_ID 58585858© 2003, Cisco Systems, Inc. All rights reserved.Presentation_ID