llnw bufferbloat
TRANSCRIPT
BufferbloatBufferbloat
Why is the Internet slow?Why is the Internet slow?
Stephen HemmingerStephen [email protected]@networkplumber.org
Limelight Networks May 2015
BufferbloatBufferbloat
● What is the problemWhat is the problem● What causes the problemWhat causes the problem● SolutionsSolutions
– DemonstrationDemonstration● Current statusCurrent status
– The GoodThe Good– The BadThe Bad– The UglyThe Ugly
Network PerformanceNetwork Performance
ThroughputMbits/sec
LatencyRound Trip Time
Fairness
How bad is it?How bad is it?
Why is worse now?Why is worse now?
● No longer in FTP universeNo longer in FTP universe● TCP Initial Window Size = 10TCP Initial Window Size = 10● Advanced TCP algorithmsAdvanced TCP algorithms● Network Offload packet trainsNetwork Offload packet trains● Bad benchmarksBad benchmarks● Router/switch memory is cheapRouter/switch memory is cheap● Everyone fears dropping packetsEveryone fears dropping packets
Head of Line blockingHead of Line blocking
Queuing Theory BasicsQueuing Theory Basics
averagetimeinqueueutilization
servicerate1 utilization
Graphic courtesy Sprint, Apricot 2004
From Fred Baker: Bufferbloat!
Netalyzr: DownstreamNetalyzr: Downstream
Netalyzr: UpstreamNetalyzr: Upstream
TCP throughput dynamicsTCP throughput dynamicsIn
crea
sing
Mea
sura
ble
Thr
ough
put
Increasing TCP Window
“knee” “cliff”
Bottleneck Capacity
QueueDepth
meanthroughput effectivewindowmeanroundtriptime
From Fred Baker: Bufferbloat!
Blame LinuxBlame Linux
● Windows XPWindows XP– Maximum window 64kMaximum window 64k
● Windows 7Windows 7– Bandwidth limit to 80 mbitsBandwidth limit to 80 mbits
● AndroidAndroid– Receive window limitedReceive window limited
Blame the customerBlame the customer
● Customers call Customers call supportsupport
● Applications are Applications are using more using more bandwidthbandwidth
● Block and chargeBlock and charge
Why Queueing?Why Queueing?
Priority Queue: TheoryPriority Queue: Theory
Priority Queue: RealityPriority Queue: Reality
Hierarchical Token BucketHierarchical Token Bucket
Random Early DetectRandom Early Detect
Explicit Congestion NotificationExplicit Congestion Notification
PACKETS
Ideal Active Queue ManagementIdeal Active Queue Management
● FairFair– All flows get some bandwidthAll flows get some bandwidth
● SimpleSimple– No tuningNo tuning
● Easy to deployEasy to deploy– No special hardware, no protocol changesNo special hardware, no protocol changes
● ReasonableReasonable– Won't create multi-second latencyWon't create multi-second latency
Stocastic Fair QueueStocastic Fair Queue
CodelCodel
PIEPIE
CakeCake
Priority
Fair Queue
CoDel
Back to Reality
Benchmark issuesBenchmark issues
● BadBad– Bytes/secBytes/sec– Packes/secPackes/sec– LatencyLatency
● GoodGood– Throughput + LatencyThroughput + Latency– Multiple connectionsMultiple connections– Real not simulatedReal not simulated
Better toolsBetter tools
● DSLreport'sDSLreport's– http://www.dslreports.com/speedtest/http://www.dslreports.com/speedtest/
● ICSI netalyzrICSI netalyzr– http://netalyzr.icsi.berkeley.edu/http://netalyzr.icsi.berkeley.edu/
● Real time Response Under LoadReal time Response Under Load– https://github.com/tohojo/netperf-wrapperhttps://github.com/tohojo/netperf-wrapper– Flent: The Flexible Network TesterFlent: The Flexible Network Tester
DSLReports speedtestDSLReports speedtest
5 sec delay!!
FIFO – the defaultFIFO – the default
Høiland-Jørgensen T., Battling Bufferbloat
Ping 1sec!
Upload
Download
Stochastic Fair QueueStochastic Fair Queue
Høiland-Jørgensen T., Battling Bufferbloat
Controlled Delay - codelControlled Delay - codel
Høiland-Jørgensen T., Battling Bufferbloat
Fair Queue Controlled DelayFair Queue Controlled Delay
Høiland-Jørgensen T., Battling Bufferbloat
Bufferbloat statusBufferbloat status
Bufferbloat: The GoodBufferbloat: The Good
● Industry awarenessIndustry awareness● Standards progressStandards progress● Active researchActive research● Open Source SolutionsOpen Source Solutions
Once you start looking,Once you start looking,Bufferbloat can be everywhereBufferbloat can be everywhere● EdgeEdge
– Home routerHome router● ProviderProvider● NetworkNetwork● AcceleratorsAccelerators
– CachesCaches– Load balancers,...Load balancers,...
● ServersServers
How is Bufferbloat Solved on Linux?How is Bufferbloat Solved on Linux?
● Queuing disciplinesQueuing disciplines– Codel, PIE, ...Codel, PIE, ...
● Linux internal Linux internal – BQL, FQ, PacingBQL, FQ, Pacing
● Enable good defaultsEnable good defaults– ecnecn– fq_codel fq_codel
Bufferbloat: the BadBufferbloat: the Bad
● Legacy equipmentLegacy equipment– OEM focus on cost not AQMOEM focus on cost not AQM
● Congestion on backboneCongestion on backbone– It's not a bug it's a feature?It's not a bug it's a feature?– Problem is political not technicalProblem is political not technical
Bufferbloat: the UglyBufferbloat: the Ugly
● WirelessWireless– AggregationAggregation– MulticastMulticast– Bad benchmarksBad benchmarks– Proprietary firmwareProprietary firmware
Questions?
Bufferbloat resources
Bufferbloat.net: http://bufferbloat.netEmail Lists: http://lists.bufferbloat.net
CeroWrt: http://www.bufferbloat.net/projects/cerowrt Other talks: http://mirrors.bufferbloat.net/Talks
Jim Gettys Blog – http://gettys.wordpress.com A big thanks to the bloat mailing list, Jim, Kathie, Van, Dave, Eric, ISC, and all the other the CeroWrt/OpenWrt contributors