tiziana ferrari diffserv deployment in the wide area: network design and testing1 diffserv...
TRANSCRIPT
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 1
Diffserv deployment in the wide area: network design and testing
Tiziana FerrariItalian National Institute for Nuclear Physics
TF-TANT task forcehttp://www.dante.net/tf-tant/
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 2
Overview
• Why diffserv?
• Test programme
• testbed
• interim results:
– classification and marking
– policing
– scheduling
• QoS measurement
• services:
– VLL
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 3
Motivation
• Why diffserv?– independent of layer 2 technologies– interoperability:
> diffserv domains National Research Networks (NRN)
> diffserv region: NRN + 1 diffserv core domain> independent diffserv implementation in each
diffserv domain – traffic aggregation– scalability– transparency to the application
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 4
Motivation (cont)
– (some) benefits:> network layer:
• VLL service, a potential replacement of ATM permanent connections
• resource allocation on congested links • QoS support in/to NRN which are not ATM
based> application layer:
• QoS support for mission critical applications (RSVP - diffserv)
e.g.: good QoS support to client-server distributed applications thanks to traffic aggregation
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 5
Diffserv and the European network model
DS domain
DS domain
DS domainDS domain
Non DS capabledomain
MPLS
NRN
NRN
TEN-155
(marking)
Markingpolicing scheduling
shaping
DS domain
NRN
NRN
NRN
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 6
Overview of test programme
• Diffserv – DSCP: class selectors, EF, AF– test of basic QoS features in diffserv platforms– parameter tuning– interoperability– services: definition, implementation, validation through test
applications (video streaming, IP telephony, distributed applications)
• diffserv and intserv (end-to-end QoS)• diffserv and MPLS • QoS measurement • Policy management
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 7
Work accomplished (June 99-Oct 99)
• network configuration
• baseline performance testing
• marking
• policing:
– functionality, token bucket parameter tuning (normal and exceed burst size) with TCP traffic
– CB-WFQ (Class Based WFQ): traffic isolation
• scheduling:
– CB-WFQ: traffic isolation, VLL service in the wide area
– SCFQ (Self Clocked Fair Queuing): EF, AF and best-effort
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 8
Work accomplished (cont)
• QoS measurement (ongoing):
– parameters: one-way delay and one-way delay variation
> LAN and WAN (VLL service implementation and validation)
– traffic generators with GPS antennas
– NTP
• definition of test services:
– point to point virtual leased line
– one-to-many virtual leased line
– resource allocation on congested links
– better than best-effort
– services for remote instrumentation control (AF)
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 9
Diffserv test network
EPFL
CSELT
Uni. Bologna
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 10
LAN layout (example)
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 11
Equipment
• Test workstations (ATM, Ethernet, Fast Ethernet, Giga Ethernet interfaces)
• traffic generators (3 SmartBits, Netcom Systems) connected to GPS rx
• ATM switch(es) in each test sites
• 2 Cabletron Ethernet switches
• DS capable routers:
– CISCO C7200 and/or C7500
– IBM 2212 and IBM 2216 (in 5 sites, 2 routers per site)
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 12
Interim Results
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 13
Classification, marking and policing
• Multi-field classification: OK
• packet marking: OK
• testing of different “exceed actions”: OK
• policing (at a configurable rate):
– token bucket: parameter tuning for TCP traffic performance optimisation
– normal burst
– exceed burst
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 14
Policing: test of exceed actions
Throughput:SWITCH: 1.20 MbpsDANTE: 0.38 Mbps (drop effect)
drop
precedence 0
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 15
Policing: TCP burst tolerance
• Normal and exceed burst tuning with TCP traffic
• single and multiple TCP streams
optimum values are functions of the rate R at which
traffic is policed, in particular:
– normal burst = 0.5 sec * R
independently of the platform
– exceed burst = 2 * normal burst
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 16
Policing: TCP burst tolerance (cont)
Throughput of 1 TCP connection (Mbps)Exceed (bytes)Normal
(bytes) 32000 48000 64000 96000 12800032000 0.98 1.23 1.23 1.25 1.2548000 1.09 1.21 1.25 12564000 1.18 1.24 1.2596000 1.24 1.25128000 1.25
Table 4: throughput of 1 TCP connection for increasing values of the normal and exceed burst size
Aggregate throughput of 5 concurrent TCP connection (Mbps)Exceed (bytes)Normal
(bytes) 32000 48000 64000 96000 12800032000 1.26 1.26 1.25 1.26 1.2548000 1.25 1.26 1.25 12664000 1.25 1.27 1.2596000 1.26 1.26128000 1.25
Table 5: throughput of 5 TCP connections for increasing values of the normal and exceed burst size
• CISCO platforms• Target rate: 1.25 Mbps
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 17
Policing: TCP burst tolerance (cont)
• IBM platforms (IBM 2212)• Small TCP burst tolerance in a policer can completely starve a TCP stream. Token bucket depth is function of the rate at which traffic is policed tuning needed
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 18
Policing: TCP burst tolerance (cont)
Bucket size(bytes)
Test length(sec)
TCP Throughput(Kbps)
2200 Connection stalled ~ 04400 60 0.97
60 35.2120 74.7180 89.8240 88.5300 95.6360 98.2420 99.3
6600
480 100.68800 300 118.911000 300 124.413200 300 124.815400 300 126.017600 300 125.364000 300 125.0
EF behaviour aggregate of TCP traffic, target rate = 163 Kbps
(IBM 2212)
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 19
Policing, two-parameter TB:optimum burst size for different traffic rates
•Small TCP burst tolerance: it can completely starve a TCP stream•optimum token bucket size: function of the policing rate
IBM2212
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 20
Scheduling and policing:two-parameter TB and premium buffer size
Recommendation: token bucket size < premium buffer size(for low packet loss probability)
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 21
Scheduling: CB-WFQ
Scheduling implementation to provide minimum bandwidth
guarantees to classes
• tested on CISCO platforms - C7200, IOS 12.0(5)T -• class definition: based on class selectors for aggregation OK
• bandwidth allocation: no starvation, dynamic allocation in case of classes which are not fully deploying the capacity guaranteed to them OK
• traffic isolation between EF BA and best-effort: – several combinations of TCP and UDP traffic
– variable number of TCP streams (different aggregation degrees)
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 22
CB-WFQ: set-up
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 23
CB-WFQ: traffic isolation, 1 TCP stream
Throughput of a single TCP high priority connection:CB-WFQ rate = 1300 Kbps
Test site (tx) Neighbour (rx) Throughput (Kbps)GRNET 1250CERN
INFN 1210CERN 700GRNET
Uni. of Utrecht 710CERN 100
Uni. of Stuttgart 100INFN
Uni. of Utrecht 100SWITCH NA
Uni. of Stuttgart 880Uni of Twente
Uni. of Utrecht 880GRNET 1110
INFN 1160Uni of Utrecht
Uni. of Twente 1180Table 6: results of CAR and WFQ tests with 1 TCP high priority stream and UDP background traffic• Inconsistent results, for some sites TCP throughput < 1.25 Mbps
• different ATM burst tolerances on the WAN connections seem seem to be the cause traffic isolation on WAN connections can be an issue
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 24
CB-WFQ: traffic isolation, multiple TCP streams
Throughput of multiple high priority TCP connections:CB-WFQ rate = 1300 Kbps, UDP background traffic at 2 Mbps
Source site Destination site Number ofTCP streams
Aggregate applicationTCP throughput (Kbps)
1 1003 130
10 221
INFN CERN
20 6801 7003 1000
10 1180
GRNET CERN
20 1270
• Overall TCP performance of the behaviour aggregate improves when the aggregation degree increases• TCP performance of a single connection still low problem under investigation
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 25
Scheduling: Self Clocked Fair Queuing
• IBM 2212
• Policy = (traffic profile, validity period, diffserv action)
• Diffserv action = (type of marking, queue type, bandwidth)
• optimum traffic isolation (tests only with UDP)
Premium TCP traffic throughput, target rate: 163 KbpsTest
numberStreams BE
throughput(Kbps)
AssuredThroughput
(Kbps)
Premiumthroughput
(Kbps)
Total throughput(Kbps)
1 BE 1967.7 / / 1967.72 A / 1968.0 / 1968.03 P / / 159.8 159.84 BE + A 649.8 1367.0 / 2016.85 BE + P 1852.5 / 159.8 2012.36 A + P / 1852.0 159.8 2011.87 BE + A + P 617.8 1236.9 159.8 2014.6
Premium: 163.8 Kbps guaranteed (8% PPP link bw)Assured: 819.2 Kbps (40% PPP bw)
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 26
QoS measurement
• Resource allocation monitoring
– for resource allocation and network dimensioning
• performance measurement
– passive
– active (invasive traffic)
– for service validation
– to understand the effect on end-to-end performance of buffering in one router or in a chain
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 27
Parameters
• parameters of interest:
– one-way delay, requirements: clock synchronisation> GPS based synchronisation
• SmartBits (Netcom Systems)> NTP (Network Time Protocol)
• NTP client / server hierarchy– one-way delay variation
> deployment of inter packet gap at rx and tx
– packet loss probability
– throughput
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 28
QoS measurement: 1-way delay and instantaneous delay variation (LAN)
SmartBitstx rx
Goal : VLL service implementation and validationMetric: Type-P-One-way-Delay (RFC 2679), type-P-one-way-ipdv Variables:
• WFQ/SCFQ EF queue depth• packet size• EF queue “weight” • traffic volume
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 29
One-way delay: average vs packet size
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 30
Latency in time
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 31
ipdv (*without* best-effort)
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 32
ipdv (*with* best-effort)
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 33
Services
• Virtual leased line:– point to point– one to many
• capacity allocation on congested links (e.g. US links)
• better-than-best-effort (qualitative definition)
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 34
VLL
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 35
VLL: implementation
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 36
Future work
• Weighted Random Early Discard (WRED) testing
• completion of IBM testing
• study of TCP related issues: ACK colouring, TCP and UDP aggregations with AF
• interoperability testing (other future platforms: NORTEL, TELEBIT, TORRENT, Linux)
• service implementation and validation
• diffserv and MPLS
• diffserv and intserv
• policy management and bandwidth brokerage
Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 37
To know more
• Activities of the TF-TANT task force:
– http://www.dante.net/tf-tant
• Diffserv test programme and interim test results:
– http://www.cnaf.infn.it/~ferrari/tfng/ds
• QoS measurement test programme:
– http://www.cnaf.infn.it/~ferrari/tfng/qosmon