DYNAMIC SERVICE CHAINING

WITH

DYSCO

Pamela Zave

AT&T Labs—Research

Ronaldo A. Ferreira

UFMS, Brazil

Xuan Kelvin Zou

Google

Masaharu Morimoto

NEC Corporation of America

Jennifer Rexford

Princeton University

forcing packets through“middleboxes” forsecurity, optimizingperformance, enhancingreachability, etc.

SERVICE CHAINING: THE STATE OF THE ART

central controllermust respond (inreal time) with ruleupdates for failures,traffic fluctuations,resource scaling

What if the flowpath changes, butpackets of existingsessions must still besent to the original boxes?(“session affinity”)

What if a box changespacket headers, sopackets coming outof the box do notmatch the forwardingrules?

What if a boxclassifies packets,to send some ona different service chain?in a data center,

each dotted line is aforwarding rule fordirecting packets ofa flow through thisservice chain

the number of rulesis on the order of(10K chain length 2)

middleboxes

switches

there has been progress on some of these problems,

but with so many of them

(5 already, 3 to go),

maybe it’s time to consider a true end-to-end approach

p1 p8

p2 p3 p4 p5 p6 p7

Dysco subsession(A, p2, B, p3)

Dysco subsession(B, p4, C, p5)

Dysco subsession(C, p6, D, p7)

middle-box

middle-box

TCPendpoint

D

TCPendpoint

A

TCP session (A, p1, D, p8)

TCP sessionlooks normal

endpoints do reliabilityand congestion control

Dyscoagenthost A

Dyscoagenthost B

Dyscoagenthost C

Dyscoagenthost D

DYSCO IS A SESSION PROTOCOL FOR SERVICE CHAINING

packets in the network arerewritten to have the 5-tuplesof their subsessions

service chain is set upas part of the TCP SYNhandshake

payload

there is little more state than whatwas already present in statefulendpoints and middleboxes

SYN [B,C,D] SYN [C,D] SYN [D]

any agent can cache policies(abstract or concrete servicechains) obtained from a policyserver

middle-box

middle-box

TCPendpoint

D

TCPendpoint

A

Dyscoagenthost A

Dyscoagenthost B

Dyscoagenthost C

Dyscoagenthost D

DYSCO AGENTS HAVE AUTONOMOUS POWER

most middleboxes rununmodified—Dysco istransparent to them

session affinity comes forfree

Dysco agents maintain the mappingbetween TCP 5-tuples and subsessions,so a middlebox that modifies the5-tuple is no problem

with an API, a middlebox can classifySYN packets and tell the Dysco agentwhere to send them next

FURTHER MOTIVATIONS

if an endpoint is multihomed(Multipath TCP), this approach makesit possible to merge all streams at onemiddlebox

DYSCO SERVICE CHAINING IS INDEPENDENT OF ROUTING

end-point

end-point

end-point

middle-box

middle-box

normal,address-basedIP forwarding

if traffic is encrypted (mcTLS), thisapproach makes it possible for Dyscoagents to exchange keys

packets of a subsession need nottake the same path in both directions

subsessions can be routedthrough middleboxes

if a middlebox is outsourcedto another cloud, thisapproach works across domain boundaries

middle-box

middle-box

middle-box

TCPendpoint

E

TCPendpoint

A

TCP session (A, E)

Dyscoagenthost B

Dyscoagenthost C

Dyscoagenthost D

INCREMENTAL AND SECURE DEPLOYMENT

SYN[E]SYN

[C,D,E]SYN[D,E] SYN[E]

service chain starts whereverSYN is routed to ahost with an agent

endpoints do not have Dysco agents

service chain extendsonly to trusted hosts

another service chain canstart among another trust group

within a trust group,signaling can beprotected bystandard techniques(nonces, encryption)

INSERT

DELETE

. . . a packet scrubber when intrusion detection raises an alarm

. . . a video transcoder during periods of network congestion

. . . a load balancer after the server has been chosen

. . . a caching proxy if the content is non-cacheable

REPLACE . . . a middlebox that needs maintenance

. . . a middlebox that has become a hairpin after endpoint mobility

DYSCO IN THE DATA CENTER

POLICY SERVERS

decide which session goes throughwhich service chain

do nothing to enforce these decisions

DYSCO AGENTS IN HOSTS

cache decisions

do all the work of enforcingthem

policy servers have an additional degree of freedom,because they can trigger

DYNAMIC RECONFIGURATION of the service chain for an ongoing session!

WHY?

HERE IS WHERE WE SHOW OUR TECHNICAL WIZARDRY:

leftanchor

rightanchor

new segmentagent agent

agent agent

agent

agent agent

PROTOCOL FOR DYNAMIC RECONFIGURATION OF A SERVICE CHAIN

old segment

GENERAL

EFFICIENT VERIFIED CORRECThandles concurrentattempts to reconfigurea session

handles failure tocreate a new segment

no packet bufferingrequired (unless oldbox state will migrateto new segment)

can delete middleboxeseven if they added/removed data (alteringsequence numbers)

TCP bytesacknowledged on thesame path on whichthey were sent

used the model-checker Spin

no data loss,deadlocks, undefined cases,unnecessary failures,inconsistent sequencenumbers,dirty terminationshandles race conditions

PROTOTYPE IMPLEMENTATION

Host A Host B Host C

Agent is a Linux kernel module

can be used with Docker, Mininet

computes incremental checksums

intercepts packets in the device driver

Daemon runs in user space

implements reconfigurationprotocol

communicates with thePolicy Server

(could alsouse DPDK)

DYSCO DEGRADES PERFORMANCE VERY LITTLE

SESSION INITIATION

TCP GOODPUT

SERVER REQUESTS PER SECOND

session initiation with 4 middleboxes

worst case: checksum computation not offloaded to NIC

average Dysco delay .094 ms

1000 sessions going through the same middlebox (link is saturated)

worst-case Dysco penalty is 1.5%

we use NGINX HTTP server

load is approximately 300,000 requests per second

4 middleboxes between the client and server

worst-case Dysco penalty is 1.8% fewer requests per second

RECONFIGURATION IMPROVES PERFORMANCE

600 TCPsessions, each goingthrough aproxy

at intervals,we trigger removal of theproxy from1/4 of thesessions

80% ofreconfigura-tions takeless than 2 ms, nonemore than4 ms

after allremovals,CPU utilization at the proxy drops to zero, GOODPUT DOUBLES

CONCLUSIONS

LIMITATIONS

Reconfigurationprotocol usesassumptions that do not always hold acrossdomain boundaries . . .

. . . mainly because ourTCP sessions needmetadata and controlsignaling.

we are not alone!

e.g., Multipath TCPhas similar problems

This is a problem cryingfor a general solution.

DYSCO IMPROVES ON . . .

. . . previous work thatuses a session protocolor encapsulation forservice chaining:

DOAConnection AcrobaticsNSH

. . . TCP-oriented protocolsfor . . .

. . . middleboxes that decrypt (mcTLS)

. . . multihoming (Multipath TCP)

. . . mobility (ECCP, TCP Migrate, msocket)

FUTURE DIRECTIONSDYSCO APPEARSCOMPATIBLE WITH . . .

Integrate these efforts,using the findings ofeach to improve theothers.

Compare directly withfine-grained forwardingrules as an approach toservice chaining.