fcp: a flexible transport framework for accommodating diversity
DESCRIPTION
FCP: A Flexible Transport Framework for Accommodating Diversity . Dongsu Han (KAIST) Robert Grandl † , Aditya Akella † , Srinivasan Seshan * † University of Wisconsin-Madison * Carnegie Mellon University. Evolution of Transport Protocols. {. Reliability/Loss recovery - PowerPoint PPT PresentationTRANSCRIPT
1
FCP: A Flexible Transport Framework for Accommodating Diversity
Dongsu Han (KAIST)Robert Grandl† , Aditya Akella†,
Srinivasan Seshan*
† University of Wisconsin-Madison*Carnegie Mellon University
2
Evolution of Transport Protocols
Network (IP)Transport
{ Reliability/Loss recoveryIn-order deliveryFlow controlCongestion control
3
Evolution of Transport Protocols
Multipath
Transport
Quality of Experience(QoE)
4
Evolution of Transport Protocols
Multipath
Transport
Quality of Experience(QoE)
Support for diversity:Multi-homing, multiple paths,
multiple sub-streams,multimedia streaming,
Web applications,Data transfer with a deadline
5
Evolution of Transport Protocols
Transport
{ Reliability/Loss recoveryIn-order deliveryFlow controlCongestion controlCongestion control
6
Evolution of Transport Protocols
Congestion control
Transport
Resource Allocation
Fairness and efficiency
Coordination
7
Problems in Supporting Diversity in Congestion Control
Cannot ensure coexistence.
RCP
TCPD3
[sigcomm]
XCP [sigcomm]
8
Evolution of TCPSACK (1996)NewReno (1999)
MulTCPExplicit Congestion Notification
TCP NiceFastTCP
TCP window-scaling, 1988
TCP BIC, CUBIC 2008
MPTCP2010
TCP Friendly Rate Control
1. End-point flexibility: Purely end-point based2. Coexistence: Invariant for fairness (TCP friendliness)
MPTCP
TFRC
NewReno
CUBIC
9
End-point based vs. Router-AssistedHighFlexibility,Diversity
End-point based [TCP]
High Efficiency
Router-Assisted [XCP, RCP]
Feedback on network’s state
Can we achieve the best of both worlds?
10
Our Approach
1. Decouple coexistence issues (fairness and efficiency) from end-point control
Flex
ible
Con
trol
Fairn
ess
Effici
ency
End-host Network
2. Introduce generic abstractions for resource allocation.
Network’s state (feedback)
End-point’s future behavior (feed-forward)
11
Flow 1
Budget W ($/sec)
Flow i’s price: Pi ($/Byte)
Sender ReceiverNetwork
...Flow 2
Flow nSender can distribute its budget to its flows. Flow i’s budget wi,
subject to W ≥ Σwi
Flow i’s rate: Ri = budget/price = wi /Pi (Byte/sec)
PRICE
Decoupling for Flexibility1
12
Flow 1
Budget W ($/sec)
Sender ReceiverNetwork
...Flow 2
Flow n
Flow i’s budget wi,
subject to W ≥ Σwi
PRICE
Decoupling for Flexibility1
1. Flexibility at the end-points in how its budget is used.
2. Flexibility in network price generation.
Invariant for fairness
13
• Feedback: “congestion price” reflecting the “cost” of sending data across the link [Kelly]
P1P2P3
Feedback: Pricing2a
Price = p3
Round Trip Time
Price = p3+p2
Round Trip Time
Price = p3+p2+p1
Round Trip Time
Congestion Header Congestion Header Congestion Header
14
• Feedback: “congestion price” reflecting the “cost” of sending data across the link [Kelly]
P1P2P3
Feedback: Pricing2a
Price feedback = p3+p2+p1
PriceEchoback
Sender updatesthe rate.Rate = budget/price
This implements proportional fairness [Kelly].
15
Price Calculation
Router updates the price, upon packet reception.
Time
Load
Averaging window = 2 x AvgRTT
Feedback: Pricing2a
Recent load
f(Average recent load)Price ($/byte) =
16
Price CalculationRouter updates the price, upon packet reception.
Time
Instantaneous Incoming budget ($)= Price ($/byte) x Bytes Received (bytes)
Averaging window = 2 x AvgRTT
Feedback: Pricing2a
Incoming budget
Average Incoming BudgetRemaining Link Capacity
Price ($/byte) =
17
Price Calculation
Time
Instantaneous Incoming budget ($)
Average Incoming BudgetRemaining Link Capacity
Price ($/byte) =
Feedback: Pricing2aNew Price
Round Trip Time (rtt)
Incoming budget I(t)=
Router stores recent history of price,
18
Feed-forward: Preloading2b
1 ($/sec) 10 ($/sec)Sender
PRICE
Price: p $/byteRate = 1/p Bps
Rate = 10/p Bps
19
Feed-forward: Preloading2b
1 ($/sec) 10 ($/sec)Sender
Rate = 1/p
Rate = 10/p’
Price
Round Trip Time
Preload=9
Price
Round Trip Time
Preload=9
(1+preload) x price(t-rtt) x bytes
Inco
min
g bu
dget
($)
Price goes up (from p to p’)
Congestion Header
20
Evaluation
Flow 1
Budget W ($/sec)
...Flow 2
Flow n
PRICE
3. Flexibility in network price generation.
2. End-point budget allocation.
1. Basic performance of FCP
21
Fast Convergence/Accurate Feedback Se
ndin
g Ra
te (M
bps)
Time
RTT = {25ms, 50ms, 125ms, 250ms, 625ms }
22
Fast Convergence/Accurate Feedback Se
ndin
g Ra
te (M
bps)
Time
Ideal RTT = {25ms, 50ms, 125ms, 250ms, 625ms }
23
Fast Convergence/Accurate Feedback Se
ndin
g Ra
te (M
bps)
Time
Ideal RTT = {25ms, 50ms, 125ms, 250ms, 625ms }
24
Fast Convergence/Accurate Feedback Se
ndin
g Ra
te (M
bps)
Time
Ideal RTT = {25ms, 50ms, 125ms, 250ms, 625ms }
25
Fast Convergence/Accurate Feedback Se
ndin
g Ra
te (M
bps)
Time
Ideal RTT = {25ms, 50ms, 125ms, 250ms, 625ms }
26
Fast Convergence/Accurate Feedback Se
ndin
g Ra
te (M
bps)
Time
Ideal RTT = {25ms, 50ms, 125ms, 250ms, 625ms }
27
Fast Convergence/Accurate Feedback
Time
FCP
Ideal
Send
ing
Rate
(Mbp
s)Se
ndin
g Ra
te (M
bps)
28
Fast Convergence/Accurate Feedback
Time
FCP
RCPIdeal
Send
ing
Rate
(Mbp
s)Se
ndin
g Ra
te (M
bps)
Send
ing
Rate
(Mbp
s)
Overloaded when new flows arrive
29
Fast Convergence/Accurate Feedback
Fairness AIMD
Time
FCP
RCP
XCP
Ideal
Send
ing
Rate
(Mbp
s)Se
ndin
g Ra
te (M
bps)
Send
ing
Rate
(Mbp
s)Se
ndin
g Ra
te (M
bps)
Overloaded when new flows arrive
30
Local Stability of FCP
• Random perturbation of [-30,30]% introduced every second for a duration of 100 ms.
31
Faster Convergence/Accurate Feedback
FCP’s converges faster and the feedback is more accurate.
Fan-
out =
5
…
100Mbps
Slower, inaccurate
Slowest. Fairness AIMD
More results in the paper:- Fairness, efficiency- Local stability- Effectiveness of preloading- Overhead of price calculation
32
Budget Determines the Fair-share.
Fan-out =50…
Host A Host B
100Mbps
1 ($/sec) 1 ($/sec)
33
Local Stability of FCP
• Random perturbation of [-30,30]% introduced every second for a duration of 100 ms.
3434
Accurate feedback with preloading• Double the load every RTT from t= 0 to 1 sec.• RCP: double the # of flows• FCP (our design): double the budget
35
Accurate feedback with preloading• Double the load every RTT from t= 0 to 1 sec.• RCP: double the # of flows• FCP (our design): double the budget
36
End-point Flexibility: Topology
Sender0 (S0)Budget 1
Sender1 (S1)Budget 2
Sender2 (S2)Budget 1
Receiver0
Receiver1
Round-trip latency : 12 ms
250 Mbps link
50 Mbps link200 Mbps links
Top flow
Btm flow
37
Budget allocation is up to end-points
Budget of S1’s top flow ($/sec)
S1’s
Thro
ughp
ut (M
bps)
Total
38
Budget allocation is up to end-points
Budget of S1’s top flow ($/sec)
S1’s
Thro
ughp
ut (M
bps)
Total Top flow Bottom flow
Max-throughput
Equal-throughput
Equal-budget
39
End-point Diversity: Background flows
Non-bottleneckLink capacity: 200Mbps
100Mbps50Mbps
Flow1 Flow2 (Background)Flow3
Flow0
Link latency: 3ms
Flow2 (Background)
40
Diversity in Network Pricing
• FCP can also support diverse behaviors in network price generation.
• Examples (in the paper)– Deadline support [D3 SIGCOMM 11]– Aggregate resource allocation in a multi-tenant
data-center– Stable bandwidth allocation for streaming– Multicast congestion control
41
Deadline Support [D3 SIGCOMM 11]
Deadline Queue
Best Effort Queue
Price
Time
Constant Pricing
Preload
Price
Time
Variable Pricing
Deadline flows preload to specify the desired rate
Average Incoming BudgetRemaining Link Capacity
Price ($/byte) =
42
Differential Pricing for Deadline Support
Constant Pricing for Deadline Flows
Pric
ePr
ice
Constant Pricing for Deadline Flows
Variable Pricingfor BE flows
43
Differential Pricing for Deadline Support
Dead flows
Rejected/BE flows
Best effort (BE)
Pric
e
Constant Pricing for Deadline Flows
Variable Pricingfor BE flows
Price increase
44
Packet processing performance
Per-core FCP throughput : 2.6Mpps (1.25 Gbps @ 64 bytes)Per-core Click baseline: 6.1 Mpps (3.1 Gbps @64bytes)
45
Packet processing performance
About 1/3 spent on actual price calculation1/3 spent on clock_gettime (can be offloaded to h/w)
46
Summary
• FCP accommodates diverse behaviors in resource allocation while utilizing explicit feedback.
• FCP maximizes end-point’s flexibility by simplifying the mechanism of coexistence.
• FCP’s explicit feed-back and feed-forward provides a generic interface for efficient resource allocation.
47
Research Theme
• Cooperative model for resource allocation.• Network knows information that end-points
do not know.• End-points know information that the network
does not know. • Can we benefit from a new interface between
the network and the end-points?
48
Any more examples?
49
Mobile Networks
• Unique properties:– Under a single administrative domain– A complete ecosystem.
50
51
• Prof. Song
52
What is research?
• People have their own style: “diversity”• What is your “uniqueness”?
Understanding Tradeoffs in Incremental Deployment of New Network Architectures CoNEXT 2013FCP: A Flexible Transport Framework for Accommodating Diversity Sigcomm 2013CAMEO: A Middleware for Mobile Advertisement Delivery MobiSys 2013Supporting Long Term Evolution in an Internet Architecture Ph.D. ThesisRPT: Re-architecting Loss Protection for Content-Aware Networks NSDI 2012XIA: Efficient Support for Evolvable Internetworking NSDI 2012
53
Conclusion
54
55
Budget allocation is up to end-points
Budget of S1’s top flow ($/sec)
S1’s
Thro
ughp
ut (M
bps)
Total Top flow Bottom flow
56
Equal-throughput
Budget allocation is up to end-points
Budget of S1’s top flow ($/sec)
S1’s
Thro
ughp
ut (M
bps)
Total Top flow Bottom flow
57
Budget Determines the Fair-share.
Fan-out =50…
Host A Host B
100Mbps
1 ($/sec) 1 ($/sec)
58
Misbehaving Hosts
Detection of misbehaving hosts: Expected incoming budget ≠ incoming budgetIdentification: Need per-host state.
59
Packet processing performance
Per-core FCP throughput : 2.6Mpps (1.25 Gbps @ 64 bytes)Per-core Click baseline: 6.1 Mpps (3.1 Gbps @64bytes)
60
Packet processing performance
About 1/3 spent on actual price calculation1/3 spent on clock_gettime (can be offloaded to h/w)
61
Memory Overhead
• Implementation stores 500 msec of history at 10 usec granularity = 50K entries
Time
Price ($/byte), Incoming Budget($)
More recent
62
Local Stability of FCP
• Random perturbation of [-30,30]% introduced every second for a duration of 100 ms.
63
Incremental Deployment
TCP Queue
Legacy TCP link
FCP link
TCP
FCP
FCP segment
1. Assign budget on aggregate TCP flows.2. TCP flows are treated as a single FCP flow.3. The mapped FCP flow’s rate determines the TCP queue’s drain rate.
FCP feedback
TCP
64
FCP and RCP
65
Short Flow’s Completion Time
6666
Accurate feedback with preloading• Double the load every RTT from t= 0 to 1 sec.• RCP: double the # of flows• FCP (our design): double the budget
67
Accurate feedback with preloading• Double the load every RTT from t= 0 to 1 sec.• RCP: double the # of flows• FCP (our design): double the budget
68
Budget Determines the Fair-share.
Fan-out =50…
Host A Host B
100Mbps
1 ($/sec) 1 ($/sec)
69
Requirements for CC Diversity (1)1) Flexible control at the end-points to handle
new requirements.
Maximize throughput with multiple pathsSupport deadlines [sigcomm 2011,2012]Support background flowsOptimize for Quality of Experience [sigcomm2013]
High-level Application Objectives
High-level Application Objectives
70
High-level Application Objectives
Requirements for CC Diversity (2)
1) Flexible control at the end-points2) Ensure coexistence (fairness and effciency)
Fairness and efficiency
71
Aggregate Control in a Multi-tenant Data-center
ToR Switch
20 Servers/Rack
VM1 VM2
VM3 VM4
VM39 VM40
…
Grou
p 0
Grou
p 1
Two tenants with the same weight sharing a rack. Each of their flow has equal budget. The network dynamically translates the value.
A rack with 20 servers
1¥/sec 20 $/sec
20 $/sec
1 ¥/secLink price: 1 €/bit
Weight 1
Weight 1
Price feedback (¥): 1 €/bit x exchange ¥ /€
Price feedback ($): 1 €/bit x exchange $/€
72
Aggregate Control in a Multi-tenant Data-center
ToR Switch
20 Servers/Rack
VM1 VM2
VM3 VM4
VM39 VM40
…
73
Feed-forward: Preloading2b 1 ($/sec)
PRICE
10 ($/sec)Sender
PRICE Budget = $10/sec
Preload = PRELOAD PRELOAD PRELOAD
PRELOAD PRELOAD PRELOAD
PRICE PRICE
PRICE PRICE
74
• Feedback: “congestion price” reflecting the “cost” of sending data across the link [Kelly]
P1P2P3
Total Price = p1+p2+p3PRICE DATA
PRICE ACK
PriceEchoback
Sender updatesthe rate.Rate = budget/price
This implements proportional fairness [Kelly].
Feedback: Pricing2a
75
Deadline Support [D3 SIGCOMM 11]
1) Flows specify the desired rate by preloading.2) The network admits a flow if it is within its capacity, otherwise the flow gets a best effort service. – Two class queuing at the network (deadline, BE)– Deadline (BE) flows get fixed (variable) pricing.
Admitted deadline flows
Rejected deadline flows
Best effort
76
Faster Convergence/Accurate Feedback
FCP’s converges faster and the feedback is more accurate.
Fan-
out =
5
…
100Mbps
Overloaded when new flows arrive
Fairness AIMD
77
Price CalculationRouter updates the price, upon packet reception.
Time
Instantaneous Incoming budget ($)= Price ($/byte) x Bytes Received (bytes)
Averaging window = 2 x AvgRTT
Average Incoming BudgetRemaining Link Capacity
Price ($/byte) =
Feedback: Pricing2a
Incoming budget I(t)=