BRKRST-3500
Designing Multipoint WAN QoS
Follow us on Twitter for real time updates of the event:
@ciscoliveeurope, #CLEUR
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 2
Housekeeping
We value your feedback- don't forget to complete your online session evaluations after each session & the Overall Conference Evaluation which will be available online from Thursday
Visit the World of Solutions and Meet the Engineer
Visit the Cisco Store to purchase your recommended readings
Please switch off your mobile phones
After the event don’t forget to visit Cisco Live Virtual: www.ciscolivevirtual.com
Follow us on Twitter for real time updates of the event: @ciscoliveeurope, #CLEUR
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 3
Bridge Puzzle
Need the flashlight to cross
Only two at a time
Fast as slowest person
Abe – 1 Minute
Bob – 2 Minutes
Chad – 5 Minutes
Dave – 6 Minutes
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 4
Bridge Puzzle
What if the slow guys walk together?
Abe + Bob (2)
Abe returns (1)
Chad + Dave (6)
Bob returns (2)
Abe + Bob (2)
Total 13 Minutes
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 5
Session Objectives
After attending this session, the participants should be able to:
Understand the challenges for Cloud, distributed Internet access, video conferencing, Unified Communication and active/active datacenter over non-QoS aware WANs
Understand available and emerging solutions to these problems
Learn how to increase visibility and control over ingress Internet bandwidth consumption
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 6
Abstract
Real-time and business critical application, such as cloud SaaS applications, Unified Communications and video, are driving the need for any-to-any connectivity with deterministic Quality of Service (QoS). This creates new challenges for multipoint wide area network (WAN) environments that are not QoS-aware, such as the Internet and DMVPN networks.
While the requirements have changed, the tools available to provide QoS in multipoint WAN environments have not. QoS policy enforcement points lack visibility into the quantity and type of traffic being received at branch and teleworker offices, forcing network designers to choose between resource underutilization or possible loss of real-time and business critical traffic.
This session will examine new methods of meeting today's QoS challenges, identify key design considerations, and review supporting case studies. It is intended for network architects and designers of corporate WAN infrastructures. An advanced understanding of QoS, WAN and virtual private network (VPN) design principles is recommended.
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 7
Multipoint WAN QoS
Aggregation Speed Mismatch
1000 Mbps
10 Mbps
1) Multipoint
2) 3rd Party
3) Non-QoS Aware
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 8
Agenda
Scenario: Teleworker QoS
Remote Ingress Shaping Theoretical Background
Implementing Remote Ingress Shaping
Proof of Concept Lab
Internet-Based Proof of Concept Lab
Putting it all together
Remote Ingress Shaping and Teleworker Revisited
Additional Use Cases
Buck’s Financial
Looking Ahead
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 11
Internet
Teleworker Overview Residential Traffic
PE
DC1 DC2
ISP
CPE
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 12
Notes from Jim Gettys (Bell Labs) on impact of bufferbloat
1-2 seconds latency, with very rapidly varying 1-2 seconds jitter
Bursts of duplicate acks; bursts of: retransmits; lots of SACK's; excessive packet drops – on long timescales
Bufferbloat
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 14
Terminology
Real-time = ―late‖ traffic has no value
Interactive = non-real-time, user waiting on data
Bulk = Neither real-time nor interactive
TCP = Elastic
Transport layer responds to delay and drops
Fine for interactive
Not for real-time
UDP = Inelastic
Transport layer does not respond to delay and drops
Must be ―governed‖
Describing behavior, not encapsulation
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 15
QoS Success Criteria
1. Protect voice and video traffic (Real-time)
2. Protect business applications
3. Meet user expectations (Interactive)
4. Utilize resources
5. Flexibility
6. Financial feasibility
7. Operational feasibility
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 16
QoS Success Criteria
1. Can I protect voice and video services from data?
2. Can I differentiate traffic to ensure business critical applications are not impacted?
3. Are applications performing as expected?
4. Does the solution utilize my available resources?
5. Can I deliver new services or change policy?
Example: Add voice or video to the network
6. Is the solution financially feasible?
7. Is the solution operationally feasible?
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 17
Available Approaches
No QoS (do nothing)
Change the topology
Force hub and spoke topology
Head-end shaping/per-tunnel QoS
Move to a QoS-aware WAN service
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 18
No QoS
Source http://www.bricklin.com/qos.htm
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 19
No QoS
Simple?
QoS is most important under adverse conditions
Can’t always throw bandwidth at the problem
Lack of QoS can delay
Adoption of new applications
Business capabilities
Can’t satisfy success criteria without it!
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 20
Force Hub and Spoke
Similar to point-to-point topologies
Implies Active/Standby
Residential/Guest traffic backhauled to hub
Hairpin of spoke-to-spoke traffic
Increases latency
Consumes hub bandwidth
Traffic is increasingly peer-to-peer
Inflexible
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 21
Head-end shaping/per-tunnel QoS
Shaping from hub to spoke
Per-tunnel
Per-Security Association (SA)
Deterministic and well understood
Great for hub and spoke
ISP/SP
Branch
Datacenter 2 Datacenter 1
ISP/SP
Per Tunnel QoS
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 22
Head-end shaping/per-tunnel QoS
Shaper has no visibility to multipoint traffic
TCP applications must go through the DC
Static reservation for spoke-to-spoke UDP
Remaining bandwidth statically divided among active datacenters
See calculations in Buck’s Financial case study
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 23
DMVPN Per Tunnel QoS (Dynamic)
! DMVPN Hub Configuration
Policy-map SHAPING-1.5MBPS
Class class-default
shape average 1500000
service-policy site
Policy-map SHAPING-1.0MBPS
Class class-default
shape average 1000000
service-policy site
interface Tunnel1
bandwidth 45000
ip address 10.0.0.1 255.255.255.0
ip nhrp map multicast dynamic
ip nhrp map group group1 service-policy output SHAPING-1.5MBPS
ip nhrp map group group2 service-policy output SHAPING-1.0MBPS
! Spoke Configuration
interface Tunnel1
bandwidth 1500
ip address 10.0.0.2 255.255.255.0
ip nhrp group group1
• Available in 12.4(22)T
• NHRP group per policy
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 24
Excellent multipoint model
QoS enforcement point has visibility to all traffic
Cooperation model with ISP/SP
Dependent on QoS configurations offered
Examples:
MPLS Services from a SP
Metro-Ethernet services
QoS-Aware WAN Services
ISP/SP
Branch
Datacenter 2 Datacenter 1
ISP/SP
QoS Aware WAN
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 25
No QoS
Per-Tunnel QoS-Aware
WAN Service
Protect Voice and
Video No No Yes
Support Business
Critical Apps Maybe Maybe Yes
Meet Performance
Expectations Maybe Maybe Yes
Utilizes Available
Resources Yes No Yes
Flexibility to deliver new
services No Yes Yes
Financially Feasible Yes Yes No
Operationally Feasible Maybe Maybe Yes
Valid Solution No No No
Solution Capabilities—Teleworker
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 26
No QoS Per-Tunnel
QoS-Aware
WAN Service
Remote
Ingress
Shaping
Protect Voice and
Video No No Yes Yes
Support Business
Critical Apps Maybe Maybe Yes Yes
Meet Performance
Expectations Maybe Maybe Yes Yes
Utilizes Available
Resources Yes No Yes Yes
Flexibility to deliver new
services No Yes Yes Yes
Financially Feasible Yes Yes No Yes
Operationally Feasible Maybe Maybe Yes Maybe
Valid Solution No No No Maybe
Solution Capabilities—Teleworker
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 29
Location of QoS
ISP/SP
Branch
Datacenter 2
ISP/SP
Datacenter 1
ISP/SP
Per Tunnel
QoS Aware WAN
QoS at Branch?
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 30
Remote Ingress Shaping
Create artificial bottleneck
Move queuing from ISP
Control delay and drops
Slow down TCP
Prioritize UDP
ISP
Branch 1
Datacenter 2
ISP
Datacenter 1
ISP
Remote Ingress Shaping
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 31
Mathis and TCP performance
http://www.linuxsa.org.au/meetings/2003-09/tcpperformance.screen.pdf
MSS Maximum Segment Size
RTT Round Trip Time
P Loss probability
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 32
Delay
Shaping puts “excess” traffic in a queue
Packets in Queue
Dela
y
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 33
TCP Loss
TCP design balance
Don’t over-run the receiver/network
Use available bandwidth
TCP will adjust to the correct rate based on delay and drops
TCP drops packets!
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 34
Bandwidth-Delay Product
Delay (RTT)
Ba
nd
wid
th
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 35
TCP Loss
There are 2 types of TCP loss
Detected by timeout (red area)
Detected by duplicate ACK (green area)
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 36
Summary
Slow TCP sessions
Preserve bandwidth-delay product
Make room for UDP
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 39
Remote Ingress Shaping
Objective
Create artificial bottleneck
Move queuing from ISP
Control delay and drops
ISP
Branch 1
Datacenter 2
ISP
Datacenter 1
ISP
Remote Ingress Shaping
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 40
Ingress Shaping
Problems
Platform Support
Classification
Solution
Shape egress in opposite direction
ISP
Branch
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 41
policy-map site
class voice
priority percent 33
class call-signaling
bandwidth percent 5
class critical-data
bandwidth percent 37
random-detect dscp-based
class class-default
bandwidth percent 25
random-detect
Remote Ingress Shaping Configuration example
policy-map shape-in
class class-default
shape average 1500000
service-policy site
interface FastEthernet0/1
Description Connection to branch LAN
service-policy output shape-in
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 42
Multiple Egress Interfaces/Networks
“LAN” Interface must
Support HQoS
See all WAN traffic
Branch ISP
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 43
Two Router Solution
Apply QoS Policy
ISP R1 R2
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 44
VRF-Lite Solution
ISP VRF1
Apply QoS Policy
On loopback cable
Branch Router
VRF2
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 45
GRE Loopback Tunnel Solution
Works prior to Hierarchical Queueing Framework (HQF)
Verified on 871W using 12.4(15)T
ISP VRF1
Apply QoS Policy
On loopback tunnel
Branch Router
Global
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 46
ip vrf outside ! Create 1 VRFs
rd 1:1
!
interface Loopback0 ! Create 2 loopback interfaces in global
ip address 10.1.3.3 255.255.255.255
interface Loopback1
ip address 10.1.3.4 255.255.255.255
!
interface Tunnel0 ! Tunnel 0 in VRF outside
ip vrf forwarding outside
ip address 10.3.3.3 255.255.255.0
tunnel source Loopback0
tunnel destination 10.1.3.4
service-policy output shaper
!
interface Tunnel1 ! Tunnel 1 in global
ip address 10.3.3.4 255.255.255.0
tunnel source Loopback1
tunnel destination 10.1.3.3
GRE Loopback Tunnel Configuration (1)
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 47
interface GigabitEthernet1/0 ! Physical interface in global table
ip address 10.0.13.3 255.255.255.0
!
interface GigabitEthernet2/0 ! Physical WAN interface in VRF outside
ip vrf forwarding outside
ip address 10.0.23.3 255.255.255.0
!
router eigrp 1
network 10.0.0.0
no auto-summary
!
address-family ipv4 vrf outside ! Create EIGRP peering between
network 10.0.0.0 ! VRF and global
no auto-summary
autonomous-system 1
exit-address-family
GRE Loopback Tunnel Configuration (2)
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 48
890 Series
• IOS 15.0 and above (No GRE Loopback Cable)
• Physical loopback cable
• More ports including 2 WAN ports
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 49
890 Series Loopback Cable Solution
ISP Global
Apply QoS Policy
On loopback cable
Branch Router
Switch Ports (FA0 to FA7)
WAN Ports (FA8 and Gig0)
Treat switch ports as 2nd box
Connect 2nd WAN port to Switch
Switch
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 50
interface FastEthernet7
Description Loopback cable to Gig 0
!
interface FastEthernet8
description WAN Interface
ip address 10.10.10.99 255.255.255.0
ip nat outside
!
interface GigabitEthernet0
ip address 10.10.100.1 255.255.255.0
ip nat inside
service-policy output shaper
!!
interface Vlan1
no ip address
Cisco 890 Loopback Cable Solution
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 51
Summary
These are tools you already know
Shape egress in opposite direction
Requires applicable interface
Support HQoS
See all WAN traffic
Shaping only at branch
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 54
Lab Requirements
TCP session emulation (PC1 and PC2)
WAN emulator (WAN)
Bandwidth constrained link (ISP to CPE2 Link)
Remote CPE (CPE2)
Head-end CPE (CPE1) (optional)
Wireshark
PC1 WAN PC2 ISP/SP CPE2 CPE1
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 55
Test 1 ISP Drops vs. Shaped Rate
Can we prevent ISP/SP drops due to a congested WAN link?
1) Yes
2) Yes, but it is not practical
3) No, you can’t
PC1 WAN PC2 ISP/SP CPE2 CPE1
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 56
ISP Drops vs. Shaped Rate
0
100
200
300
400
500
600
10 9.9 9.8 9.7 9.6 9.5 9.4 9.3 9.2 9.1 9 8.9 8.8 8.7 8.6 8.5 8.4 8.3 8.2 8.1 8
Dro
pp
ed
Packe
ts
Shaped Rate (Mbps)
ISP Drops
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 57
Test 2 UDP Delay and Jitter vs. Shaped Rate
Can we bound the jitter of UDP to acceptable levels under congestion?
1) Yes
2) No
PC1 WAN PC2 ISP/SP CPE2 CPE1
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 58
UDP Jitter vs. Shaped Rate
20
30
40
50
60
70
80
90
10 9.9 9.8 9.7 9.6 9.5 9.4 9.3 9.2 9.1 9 8.9 8.8 8.7 8.6 8.5 8.4 8.3 8.2 8.1 8
Jit
ter
(ms)
Shaped Rate (Mbps)
Jitter
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 59
UDP Delay vs. Shaped Rate
40
60
80
100
120
140
160
180
200
220
240
10 9.9 9.8 9.7 9.6 9.5 9.4 9.3 9.2 9.1 9 8.9 8.8 8.7 8.6 8.5 8.4 8.3 8.2 8.1 8
Avera
ge D
ela
y (
ms)
Shaped Rate (Mbps)
Average Delay
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 60
Test 3 UDP Delay and Jitter vs. TCP Sessions
How does the number of TCP sessions affect UDP delay, loss and jitter?
1) No impact
2) Low impact, no action required
3) High impact, action required
PC1 WAN PC2 ISP/SP CPE2 CPE1
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 61
UDP Average Delay vs. TCP Sessions
20
70
120
170
220
270
1 2 3 4 5 10 15 20 25 30 35 40 45 50 55 60 65 70 100
Avera
ge D
ela
y (
ms)
TCP Sessions
Average Delay
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 62
Test 4 TCP Sessions and Queue Depth
How does the number of TCP sessions affect average queue depth?
1) Hard to tell
2) No impact
3) Increases queue depth
4) Decreases queue depth
PC1 WAN PC2 ISP/SP CPE2 CPE1
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 63
Queue Depth vs. TCP Sessions
40
140
240
340
440
540
640
740
840
35 40 45 50 55 60 65 70Av
era
ge Q
ueu
e D
ep
th (
Packets
)
TCP Sessions
Average Queue Depth
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 64
Test 5 Queue Depth and UDP Delay
Will increasing queue size affect UDP delay, loss and jitter?
Yes
No
PC1 WAN PC2 ISP/SP CPE2 CPE1
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 65
Delay vs. Queue Depth
Max Queue Size (Packets) Min Delay (ms) Max Delay (ms) Avg Delay (ms)
40 48 109 70
4000 9 57 29
Difference 39 52 41
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 66
Conclusions
RIS can move queuing from ISP and reduce drops
UDP delay and jitter can be bounded to acceptable levels
Two key “knobs”
Shaped Rate – How aggressively we queue TCP packets
Queue Depth – Conserving the bandwidth delay product requires that queue depth increase linearly with the number of TCP sessions
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 68
Lab Setup
871W
3 Mbps cable Internet
ICMP RTT of 40 ms
Load generation
FTP
HTTrack
High definition Internet video
ISP VRF1
Apply QoS Policy
On loopback tunnel
Branch Router
Global
Internet
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 69
Audience Questions
Does ISP queuing delay have a significant impact on delay?
Yes
No
What is the required ingress shaped rate?
70% of line rate
80% of line rate
90% of line rate
How deep will queues need to be?
500 packets
250 packets
100 packets
40 packets
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 70
Internet-Based Tests Jitter vs. Shaped Rate
0
20
40
60
80
100
120
140
160
180
200
3.5 3.4 3.3 3.2 3.1 3 2.9 2.8 2.7 2.6 2.5 2.4 2.3 2.2 2.1 2 1.9 1.8 1.7 1.6 1.5
Jit
ter
(ms)
Shaped Rate (Mbps)
Jitter
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 71
Internet-Based Test Average Delay vs. Shaped Rate
50
55
60
65
70
75
80
85
90
95
100
3.5 3.4 3.3 3.2 3.1 3 2.9 2.8 2.7 2.6 2.5 2.4 2.3 2.2 2.1 2 1.9 1.8 1.7 1.6 1.5
Dela
y (
ms)
Shaped Rate (Mbps)
Average Delay
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 72
Conclusions
ISP queue delay peak was 55 ms
(95 ms–40 ms = 55 ms)
Nearly quadrupled one-way delay from 20 ms to 75 ms
95% of line rate
Default (40 packets) queue depth
30 ms or less average delay for real-time traffic added by branch and ISP WAN connection
GRE Loopback Tunnel on 871W with BVI
15% CPU
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 73
What Does Remote Ingress Shaping (RIS) Enable?
Two new capabilities that define the use cases
1.Allows you to maintain control over TCP applications, even if the traffic does not go through your datacenter Examples:
Cloud services (SaaS, IaaS)
Teleworkers (residential traffic)
Guest networking
Split-tunneling
2.Allows a single point of configuration and policy enforcement for a location or WAN link
Examples: A/A Datacenter
Internet Edge
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 74
Putting it all Together
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 76
Internet
Buck’s Financial Overview
Financial services company
1000s of very small branch offices
Dual datacenters
Migrating from MPLS VPN to DMVPN
DSL and broadband cable connections
Future VoIP
Branch Office
Datacenter 1 Datacenter 2
PE
ISP
3rd Party 3rd Party
ISP ISP
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 77
Internet
Buck’s Financial Challenges
Wants to leverage 3rd party (cloud) for live video
Branch owners want to use available broadband capacity
ScanSafe
Future services
GuestNet
Other 3rd parties
Branch Office
Datacenter 1 Datacenter 2
PE
3rd Party 3rd Party
ISP ISP
ISP
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 78
Head-End Shaping as a Solution
Shaper has no visibility to multipoint traffic
TCP applications must go through the DC
Static reservation for spoke-to-spoke UDP
Remaining bandwidth statically divided among active datacenters
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 79
Head-End Shaping as a Solution
Configure per-tunnel traffic shaping at each DC
720 Kbps reserved for 3rd party video (600 Kbps + 20%)
160 Kbps reserved for 2 VoIP phone calls
Remaining bandwidth divided between 2 DCs
Branch BW
3rd Party Video 2 VoIP Calls Available to DC
1.5 Mbps 720 Kbps 160 Kbps 310 Kbps
2 Mbps 720 Kbps 160 Kbps 810 Kbps
3 Mbps 720 Kbps 160 Kbps 1310 Kbps
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 80
No QoS Per-Tunnel
QoS-Aware
WAN Service
Remote
Ingress
Shaping
Protect Voice and
Video No Yes Yes Yes
Support Business
Critical Apps No Yes Yes Yes
Meet Performance
Expectations Maybe Maybe Yes Yes
Utilizes Available
Resources Yes No Yes Yes
Flexibility to deliver new
services Maybe No Maybe Yes
Financially Feasible Yes Yes No Yes
Operationally Feasible Maybe Yes Yes Maybe
Valid Solution No No No Maybe
Solution Capabilities—Buck’s Financial
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 82
Internet Edge
More than just Internet
Business-to-Business VPN
Corporate E-Commerce
Access to Cloud Services
Branch site-to-site VPN
Teleworker
User Internet access
Critical applications separated by circuits
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 83
Internet Edge
Simplified classification
Ports/Protocols works better
TCP session scaling important!
Buffering is key
Additional Tools
Ironport Web Security Appliance (WSA)
Services Control Engine (SCE)
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 84
WSA Bandwidth Controls for Streaming Media
New in WSA AsyncOS 7.0
Overall bandwidth limit.
User bandwidth limit.
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 85
Services Control Engine (SCE)
Application-layer deep packet inspection
Real-time traffic control
Granular bandwidth metering and shaping
Quota management
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 86
Explicit Congestion Notification (ECN)
Notify sender of congestion without packet loss
Specified as RFC 3186 (2001)
Requires support on hosts and network
Not widely used
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 87
Explicit Congestion Notification (ECN)
Supported in IOS since 12.2T
Disabled by default on
Windows 7
Windows Server 2008
Windows Vista
Mac OS X 10.5 and 10.6
Server Mode for
Linux
policy-map QoS_Policy
class class-default
bandwidth per 70
random-detect
random-detect ecn
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 88
RSVP
RSVP implementation could be modified to address the problem for private WANs
Requires routers to initiate reservations
See backup slides
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 89
Additional RIS Considerations
L2 and L3 overhead accounting
CPU requirements
WAAS
“Measure” optimized traffic
Transport Flow Optimization (TFO)
Viruses/scavenger class
User-Based Rate Limiting
Drop
Anti-replay
Use caution if applying QoS policies to encrypted traffic
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 91
Summary
Now you have a new tool!
RIS can overcome challenges with
Multipoint
3rd Party
Non-QoS Aware WAN
Enables acceptable UDP performance
Even if applications do not go through the DC
With a single point of configuration and policy enforcement
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST- 3500 93
Please complete your Session Survey
Don't forget to complete your online session evaluations after each session.
Complete 4 session evaluations & the Overall Conference Evaluation
(available from Thursday) to receive your Cisco Live T-shirt
Surveys can be found on the Attendee Website at www.ciscolivelondon.com/onsite
which can also be accessed through the screens at the Communication Stations
Or use the Cisco Live Mobile App to complete the
surveys from your phone, download the app at
www.ciscolivelondon.com/connect/mobile/app.html
We value your feedback
http://m.cisco.com/mat/cleu12/
1. Scan the QR code
(Go to http://tinyurl.com/qrmelist for QR code reader
software, alternatively type in the access URL above)
2. Download the app or access the mobile site
3. Log in to complete and submit the evaluations
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 96
QoS Golden Rules
Start with the goal in mind
There is no substitute for sufficient bandwidth
Queuing and Scheduling can protect voice and video from data
Only Call Admission Control can protect voice from voice and video from video
Don’t mix UDP and TCP in the same class
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 97
UDP
UDP does not adjust to loss or delay
UDP is generally only used for real-time traffic where drops are preferred to delays
DNS
Voice
Video (VC and live broadcasts)
Financial applications (ticker)
Video games
Multicast (non-real time) Content distribution
IPSec NAT-T
Does not count
Treat like TCP?
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 98
ECN Bits
2 bits in IP Header
2 bits in TCP Header
ECN-echo (ECE)
Congestion Window Reduced (CWR)
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 99
ECN
How it works
ECN negotiated during TCP handshake
Sender sets IP ECT bit
Congested router sets IP CE bit
Receiver sets TCP ECE bit (echo)
Sender receives echo
Sender acts like packet was dropped
Sender acknowledges echo (CWR)
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 100
Jitter vs. Shaped Rate
20
40
60
80
100
120
140
8.8 8.7 8.6 8.5 8.4 8.3 8.2 8 7.9 7.8 7.7 7.6 7.5 7.4 7.3 7.2
Jitter
50 TCP Sessions
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 101
Delay vs. Shaped Rate
40
60
80
100
120
140
160
180
8.8 8.7 8.6 8.5 8.4 8.3 8.2 8 7.9 7.8 7.7 7.6 7.5 7.4 7.3 7.2
Average Delay
50 TCP Sessions
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 102
TCP Only Network
ISP
Apply QoS Policy
TCP and UDP on separate interfaces
Simple configuration
Shape TCP traffic
“Reserve” bandwidth for UDP
Branch
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 103
RSVP
RSVP implementation could be modified to address the problem for private WANs
Requires routers to initiate reservations
RSVP agent
RSVP and IOS
RSVP proxy
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 104
RSVP RSVP and QoS in Cisco IOS Routers
Scheduling + Policing
Call Admission Control
? YES
NO
RS
VP
RSVP signaling
LLQ
/ C
BW
FQ
IntServ
model
Data
Control Plane
Data Plane
RS
VP
IntServ/
DiffServ
model
Scheduling + Policing
Call Admission Control
? YES
NO
Data
Control Plane
Data Plane
RSVP signaling
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 105
RSVP IntServ/DiffServ—IOS Model Interface Queuing
“U
sa
ble” B
and
wid
th (7
5%
) R
ese
rve
d
To
tal L
ink B
an
dw
idth
0%
25%
50%
75%
100%
Priority (33% max)
BW
Assig
ned to
LLQ
Cla
sses
ip rs
vp
bandw
idth
RSVP flows admitted/
rejected based on ‘ip
rsvp bandwidth’ only
RSVP flows assigned to
priority queue based on
LLQ classes
(typically, DSCP)
BW reserved for LLQ/
CBWFQ classes based
on policy maps and
service policy
Packets assigned to
LLQ classes/queues
based on class maps
(typically, DSCP)
Provision priority
queue to match
RSVP bandwidth +
L2 overhead
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 106
RSVP IntServ/DiffServ Cisco IOS Model: Notes
LLQ/CBWFQ classes can be configured as usual and bandwidth allocated to them on the interface
No bandwidth is reserved with ip rsvp bandwidth
Reservations accepted/rejected based exclusively on value configured in ip rsvp bandwidth
RSVP traffic assigned to queues based on LLQ rules (RSVP is not involved in classification)
If non-RSVP real-time applications are present, provision the PQ accordingly and ensure they use a CAC mechanism to avoid oversubscription
ip rsvp resource-provider none
ip rsvp data-packet classification none To enable this
model in IOS:
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 107
class-map match-all VOICE
match ip dscp ef ! All voice bearer traffic is marked EF
class-map match-any CALL-SIGNALING
match ip dscp cs3 ! All call signaling traffic is marked CS3
!
policy-map WAN-EDGE
class VOICE
priority percent 33 ! For Se1/0 512kbps at L2 = 18 G.729 calls
class CALL-SIGNALING
bandwidth percent 5 ! For Se1/0 77kbps = ~300 SCCP phones
!
interface Multilink1
service-policy output WAN-EDGE ! Attaches the MQC policy to Mu1
ppp multilink
ppp multilink group 1
!
interface Serial1/0
bandwidth 1536 ! Overall L2 bandwidth for this interface
ip rsvp bandwidth 448 ! RSVP BW (L3) to allow 18 G.729 calls
ip rsvp resource-provider none ! Enables IntServ/DiffServ mode
ip rsvp data-packet classification none ! Enables IntServ/DiffServ mode
ip rsvp signaling dscp 24 ! Marks RSVP signaling with DSCP CS3
no ip address
RSVP Cisco IOS Configuration Example (IntServ/DiffServ)
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 109
Happy Health Overview
Healthcare provider
MPLS VPN
Dozens of large sites
DS-3 or better
Applications
VoIP
Medical Imaging
Applications in multiple DCs
Location 1
PE
Datacenter 1
PE
Datacenter 2
PE
DR Site
PE
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 110
Happy Health Challenges
MPLS VPN Service Provider charges for “burst” usage above 50% of line rate
Location 1
PE
Datacenter 1
PE
Datacenter 2
PE
DR Site
PE
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 111
Without RIS
1) TCP applications must go through the DC (or similar QoS enforcement point) to prevent oversubscription
2) Every active datacenter must share bandwidth with other active datacenters
3) Bandwidth must be statically reserved for UDP applications that do not go through the datacenter
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 112
Egress Shaping as a Solution No Tunnels
Identify destination networks
Shape traffic toward each destination
Requires a mapping of every network to every location
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 113
ip access-list extended site1
permit ip 10.0.1.0 0.0.0.255 any
permit ip any 10.0.1.0 0.0.0.255
ip access-list extended site2
permit ip 10.0.2.0 0.0.0.255 any
permit ip any 10.0.2.0 0.0.0.255
ip access-list extended site3
permit ip 10.0.3.0 0.0.0.255 any
permit ip any 10.0.3.0 0.0.0.255
Traffic Shaping Configuration Example No Tunnels (1)
class-map match-any site1
match access-group name site1
class-map match-any site2
match access-group name site2
class-map match-any site3
match access-group name site3
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 114
policy-map site
class voice
priority percent 33
class call-signaling
bandwidth percent 5
class critical-data
bandwidth percent 37
random-detect dscp-based
class class-default
bandwidth percent 25
random-detect
Traffic Shaping Configuration Example No Tunnels (2)
policy-map all-sites
class site1
shape average 600000
service-policy site
class site2
shape average 400000
service-policy site
class site3
shape average 200000
service-policy site
interface FastEthernet0/1
service-policy output all-sites
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 115
Egress Shaping as a Solution Static Tunnels
Simplifies classification of destination networks
Requires a full-mesh overlay on top of existing any-to-any network (5050 tunnels)
Shape traffic toward each destination
Full mesh routing protocol can cause network meltdown
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 116
policy-map site
! Omitted for brevity
Traffic Shaping Configuration Example Static GRE Tunnels
policy-map 600ksite
class class-default
shape average 600000
service-policy site
policy-map 400ksite
class class-default
shape average 400000
service-policy site
Interface tunnel 1
Description tunnel to site1
service-policy output 600ksite
Interface tunnel 2
Description tunnel to site2
service-policy output 400ksite
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 117
Egress Shaping as a Solution DMVPN
Further simplifies the configuration by automating tunnel creation
New dynamic per-tunnel QoS, 12.4(22)T
Within the tunnel interface associate the QoS policy with the “ip nhrp map group” command
Simplifies the association of a QoS policy at the hub to each spoke location
http://www.cisco.com/en/US/docs/ios/security/configuration/guide/sec_per_tunnel_ qos.html#wp1072822
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 118
Traffic Shaping Configuration Example DMVPN Per Tunnel QoS (Dynamic)
Policy-map SHAPING-1.5MBPS
Class class-default
shape average 1500000
service-policy site
Policy-map SHAPING-1.0MBPS
Class class-default
shape average 1000000
service-policy site
interface Tunnel1
bandwidth 45000
ip address 10.0.0.1 255.255.255.0
ip nhrp map multicast dynamic
ip nhrp map group group1 service-policy output SHAPING-1.5MBPS
ip nhrp map group group2 service-policy output SHAPING-1.0MBPS
.
no ip mroute-cache
tunnel source 172.17.0.1
tunnel mode gre multipoint
tunnel key 253
tunnel protection ipsec profile DMVPN
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 119
No QoS
(Do Nothing) Per-Tunnel
QoS-Aware
WAN Service
Remote
Ingress
Shaping
Protect Voice and
Video Yes Yes Yes
Support Business
Critical Apps Yes Yes Yes
Meet Performance
Expectations Yes Maybe Yes
Utilizes Available
Resources Yes No Yes
Flexibility to deliver new
services Maybe Maybe Yes
Financially Feasible No Yes Yes
Operationally Feasible Yes Maybe Maybe
Valid Solution No No N/A Maybe
Solution Capabilities—Happy Health
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 120
870 Series
Loopback Cable Solution would consume 2 of 4 available LAN ports
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 121
GRE Loopback Tunnel Solution
Works prior to Hierarchical Queueing Framework (HQF)
Verified on 12.4(15)T
ISP VRF1
Apply QoS Policy
On loopback tunnel
Branch Router
VRF2
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 122
ip vrf inside
rd 2:2
ip vrf outside
rd 1:1
GRE Loopback Tunnel Configuration Two VRFs (1)
interface Loopback0
ip address 10.1.3.3 255.255.255.255
interface Loopback1
ip address 10.1.3.4 255.255.255.255
!
interface Tunnel0
ip vrf forwarding outside
ip address 10.3.3.3 255.255.255.0
tunnel source Loopback0
tunnel destination 10.1.3.4
service-policy output shape-in
interface Tunnel1
ip vrf forwarding inside
ip address 10.3.3.4 255.255.255.0
tunnel source Loopback1
tunnel destination 10.1.3.3
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 123
interface GigabitEthernet1/0
ip vrf forwarding inside
ip address 10.0.13.3 255.255.255.0
interface GigabitEthernet2/0
ip vrf forwarding outside
ip address 10.0.23.3 255.255.255.0
GRE Loopback Tunnel Configuration Two VRFs (2)
router eigrp 1
network 10.0.0.0
no auto-summary
!
address-family ipv4 vrf outside
network 10.0.0.0
no auto-summary
autonomous-system 1
exit-address-family
!
address-family ipv4 vrf inside
network 10.0.0.0
no auto-summary
autonomous-system 1
exit-address-family
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 124
GRE Loopback Tunnel Solution Single VRF and Global Table
Works prior to Hierarchical Queueing Framework (HQF)
Verified on 12.4(15)T
ISP VRF1
Apply QoS Policy
On loopback tunnel
Branch Router
Global
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 125
ip vrf outside ! Create 1 VRFs
rd 1:1
!
interface Loopback0 ! Create 2 loopback interfaces in global
ip address 10.1.3.3 255.255.255.255
interface Loopback1
ip address 10.1.3.4 255.255.255.255
!
interface Tunnel0 ! Tunnel 0 in VRF outside
ip vrf forwarding outside
ip address 10.3.3.3 255.255.255.0
tunnel source Loopback0
tunnel destination 10.1.3.4
service-policy output shaper
!
interface Tunnel1 ! Tunnel 1 in global
ip address 10.3.3.4 255.255.255.0
tunnel source Loopback1
tunnel destination 10.1.3.3
GRE Loopback Tunnel Configuration VRF and Global (1)
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 126
interface GigabitEthernet1/0 ! Physical interface in global table
ip address 10.0.13.3 255.255.255.0
!
interface GigabitEthernet2/0 ! Physical WAN interface in VRF outside
ip vrf forwarding outside
ip address 10.0.23.3 255.255.255.0
!
router eigrp 1
network 10.0.0.0
no auto-summary
!
address-family ipv4 vrf outside ! Create EIGRP peering between VRF
network 10.0.0.0 ! VRF and global
no auto-summary
autonomous-system 1
exit-address-family
GRE Loopback Tunnel Configuration VRF and Global (2)
© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-3500 127
Notes on Buffer Bloat
Gibbens and Kelly (1999)
Virtual queues 90-95% of the link capacity
No delay added
Srikant et al
AVQ – Adaptive version of virtual queue concept
Gettys
TCP will fill any buffer just before the choke point of the path
TCP design assumes that congestion will generate timely notification by packet loss or ECN
Some small amount of timely packet loss is normal and essential