a proposal of the openflow controller development support tool
DESCRIPTION
OpenFlow controller programmer does not have any method to confirm how reflected their code to the flow control, directly and intuitively “This code, how does work on... which flow?” “This flow, which code does make it?” This slides shows the basic design of the mechanism for binding code and flow to see them. It enables cross referencing logic and flow each other and also enable tracing the flow over switches. It had been presented at the 16th IOT conference of IPSJ, March 2012.TRANSCRIPT
A proposal of the OpenFlow controller development support tool - using Logic and Flow ID label -
Yutaka Yasuda, Kyoto Sangyo University
Introduction
•OpenFlow
Switch control mechanism by software program
•Problem of controller programming
Programmer does not have any method to confirm how reflected their code to the flow control, directly and intuitively
“This code, how does work on... which flow?”“This flow, which code does make it?”
host1
host4host2
host3sw1 sw3
sw2 sw4
Goal of today : development support tool proposal
Controller
• bind code to flow control information• enable to trace the affected flow
step 1: Binding Code and Flow Entry
ex. : simple 2 ports switch and typical code for that
def packet_in_callback(dpid, inport, reason, len, bufid, packet): flow = extract_flow(packet) flow[core.IN_PORT] = inport if inport == 1 :
1 2(inport=1) [output=2]
(inport=2) [output=1]convention of here : (match fields) [actions]
NOX like pseudo code:callback
make match fields
set flow entry
make actions
(inport=1) [output=2]
(inport=2) [output=1]
outport = 2 else: outport = 1 actions = [[openflow.OFPAT_OUTPUT, [0, outport]]] inst.install_datapath_flow(dpid, flow,..., actions, ...) return CONTINUE
Logic ID label setting
(inport=1) [output=2]
(inport=2) [output=1]
def packet_in_callback(dpid, inport, reason, len, bufid, packet): flow = extract_flow(packet) flow[core.IN_PORT] = inport if inport == 1 : outport = 2 else: outport = 1 actions = [[openflow.OFPAT_OUTPUT, [0, outport]]] inst.install_datapath_flow(dpid, flow,..., actions, ...) return CONTINUE
You can see the correspondence relation between code and flow entry
Logic ID label setting
(inport=1) [output=2]
(inport=2) [output=1]
def packet_in_callback(dpid, inport, reason, len, bufid, packet): flow = extract_flow(packet) flow[core.IN_PORT] = inport if inport == 1 :
outport = 2 else:
outport = 1 actions = [[openflow.OFPAT_OUTPUT, [0, outport]]] inst.install_datapath_flow(dpid, flow,..., actions, ...) return CONTINUE
inst.markLogicPoint( 101 )
inst.markLogicPoint( 102 )
#101
#102
Insert the function to set the Logic ID label to flow entry
Logic ID label setting
(inport=1) [output=2]
(inport=2) [output=1]
def packet_in_callback(dpid, inport, reason, len, bufid, packet): flow = extract_flow(packet) flow[core.IN_PORT] = inport if inport == 1 :
outport = 2 else:
outport = 1 actions = [[openflow.OFPAT_OUTPUT, [0, outport]]] inst.install_datapath_flow(dpid, flow,..., actions, ...) return CONTINUE
inst.markLogicPoint( 101 )
inst.markLogicPoint( 102 )
#101
#102
Match Fields Counters Actionsflow entry: cookie(64bits)
( inport=1 ) [ outport=2 ]#102
Cookie field is good to keep the label information
Result : cross reference of code and flow entriesswitch
def getOutport(dpid, vport, packet): if not OPORTS.has_key(dpid): log.err('Unknown dpid=%x on getoutput' % (dpid),system='paniersw') return False if not OPORTS[dpid].has_key(vport): log.err('Invalid (dpid,vport)=(%x,%d) on getOutport' % (dpid, vport),system='paniersw') return False ops=OPORTS[dpid][vport] for (rule, outport, cookie) in ops: if rule == RULE80 and testTCPport(80, packet): inst.setLogicMark( 101 ) return (outport, cookie) elif rule == RULE443 and testTCPport(443, packet): inst.setLogicMark( 102 ) return (outport, cookie ) elif rule == RULE25 and testTCPport(25, packet): return (outport, cookie) elif rule == ANY: return (outport, cookie) return False
def forward_l2_packet(dpid, inport, vport, packet, buf, bufid): dstaddr = packet.dst.tostring() if not ord(dstaddr[0]) & 1 and inst.st[dpid].has_key(dstaddr): prt = inst.st[dpid][dstaddr] if prt[0] == vport: inst.setLogicMark( 122 ) log.err('**warning** learned port = inport', system="paniersw") myFlood(dpid, bufid, buf, vport, packet) else: inst.setLogicMark( 123 ) log.msg('installing flow for ' + str(packet), system="paniersw") flow = extract_flow(packet) flow[core.IN_PORT] = inport (outport, cookie) = getOutport(dpid, prt[0], packet) actions = [[openflow.OFPAT_OUTPUT, [0, outport]]] inst.install_datapath_flow(dpid, flow, CACHE_TIMEOUT, openflow.OFP_FLOW_PERMANENT, actions, bufid, openflow.OFP_DEFAULT_PRIORITY, inport, buf, cookie) else: inst.setLogicMark( 124 ) myFlood(dpid, bufid, buf, vport, packet)
flow entries
controller program
Reference from flow to code can tell : “This flow entry comes from this code”
Reference from code to flow can tell : “Which flow entries are affected by this code”
1 6
2
3
4
5
7
8
9
10
cookie match fields, etc..
(inport=1) [ outport=6 ]
101 …………………
…………………
…………………
…………………
flow entries
Single branch : OKswitch
def getOutport(dpid, vport, packet): if not OPORTS.has_key(dpid): log.err('Unknown dpid=%x on getoutput' % (dpid),system='paniersw') return False if not OPORTS[dpid].has_key(vport): log.err('Invalid (dpid,vport)=(%x,%d) on getOutport' % (dpid, vport),system='paniersw') return False ops=OPORTS[dpid][vport] for (rule, outport, cookie) in ops: if rule == RULE80 and testTCPport(80, packet): inst.setLogicMark( 101 ) return (outport, cookie) elif rule == RULE443 and testTCPport(443, packet): inst.setLogicMark( 102 ) return (outport, cookie ) elif rule == RULE25 and testTCPport(25, packet): return (outport, cookie) elif rule == ANY: return (outport, cookie) return False
def forward_l2_packet(dpid, inport, vport, packet, buf, bufid): dstaddr = packet.dst.tostring() if not ord(dstaddr[0]) & 1 and inst.st[dpid].has_key(dstaddr): prt = inst.st[dpid][dstaddr] if prt[0] == vport: inst.setLogicMark( 122 ) log.err('**warning** learned port = inport', system="paniersw") myFlood(dpid, bufid, buf, vport, packet) else: inst.setLogicMark( 123 ) log.msg('installing flow for ' + str(packet), system="paniersw") flow = extract_flow(packet) flow[core.IN_PORT] = inport (outport, cookie) = getOutport(dpid, prt[0], packet) actions = [[openflow.OFPAT_OUTPUT, [0, outport]]] inst.install_datapath_flow(dpid, flow, CACHE_TIMEOUT, openflow.OFP_FLOW_PERMANENT, actions, bufid, openflow.OFP_DEFAULT_PRIORITY, inport, buf, cookie) else: inst.setLogicMark( 124 ) myFlood(dpid, bufid, buf, vport, packet)
flow entries
controller program
A recorded Logic ID directly means the branch path that the code actually passed through.(Programmer puts the Logic ID record function to the important and memorable points of the code. )
1 6
2
3
4
5
7
8
9
10
cookie match fields, etc..
(inport=1) [ outport=6 ]
101 …………………
…………………
…………………
…………………
flow entries
Multiple branches : needs the past recorddef getOutport(dpid, vport, packet): if not OPORTS.has_key(dpid): log.err('Unknown dpid=%x on getoutput' % (dpid),system='paniersw') return False if not OPORTS[dpid].has_key(vport): log.err('Invalid (dpid,vport)=(%x,%d) on getOutport' % (dpid, vport),system='paniersw') return False ops=OPORTS[dpid][vport] for (rule, outport, cookie) in ops: if rule == RULE80 and testTCPport(80, packet): inst.setLogicMark( 101 ) return (outport, cookie) elif rule == RULE443 and testTCPport(443, packet): inst.setLogicMark( 102 ) return (outport, cookie ) elif rule == RULE25 and testTCPport(25, packet): return (outport, cookie) elif rule == ANY: return (outport, cookie) return False
def forward_l2_packet(dpid, inport, vport, packet, buf, bufid): dstaddr = packet.dst.tostring() if not ord(dstaddr[0]) & 1 and inst.st[dpid].has_key(dstaddr): prt = inst.st[dpid][dstaddr] if prt[0] == vport: inst.setLogicMark( 122 ) log.err('**warning** learned port = inport', system="paniersw") myFlood(dpid, bufid, buf, vport, packet) else: inst.setLogicMark( 123 ) log.msg('installing flow for ' + str(packet), system="paniersw") flow = extract_flow(packet) flow[core.IN_PORT] = inport (outport, cookie) = getOutport(dpid, prt[0], packet) actions = [[openflow.OFPAT_OUTPUT, [0, outport]]] inst.install_datapath_flow(dpid, flow, CACHE_TIMEOUT, openflow.OFP_FLOW_PERMANENT, actions, bufid, openflow.OFP_DEFAULT_PRIORITY, inport, buf, cookie) else: inst.setLogicMark( 124 ) myFlood(dpid, bufid, buf, vport, packet)
flow entries1 6
2
3
4
5
7
8
9
10
cookie match fields, etc..
(inport=1) [ outport=6 ]
101 …………………
…………………
…………………
…………………
flow entries 101102
122123
125123
If memorable points are (multiple) exists and located them the isolated branch, it needs to record the both of them. (in right figure, 101 and 123 are.)It is not good to overwrite by the recent single label (123).
overwriting
Queuing for multiple label recording
flow entries1 6
2
3
4
5
7
8
9
10
cookie match fields, etc..
(inport=1) [ outport=6 ]
101 …………………
…………………
…………………
…………………
flow entries
123
1 6
2
3
4
5
7
8
9
10
cookie match fields, etc..
(inport=1) [ outport=6 ]
101,123 …………………
…………………
102,124 …………………
…………………
... ... ... ... ... ... 101 123
123
queuing38
(8 x 8 bits)
To keep the past record, Logic ID label will be queuing to Cookie field (64bits). The right figure shows the typical case, 8bits label queuing in 8 stages. The combination of bit width and the number of stages are flexible to fit the target controller program.
overwriting queuing
overwriting
Restrictions : binding code and flow entry
•Some controller SDKs may use Cookie for their purpose
Floodlight uses 32bits (12bit AppID, 20bit reserve)
• Increase switch loads to collect information
“In short, the existing statistics mechanisms are (both) relatively high overhead” (J. Mogul et al., 2010)
J. Mogul et al. “DevoFlow: Cost-Effective Flow Management for High Performance Enterprise Networks”, In Proc. ACM SIGCOMM HotNets-IX, 2010
step 2: Tracing Flow Over the Switch
Flow tracing (idea)
•Demand : want to see what happen on the flow from here on
Now, flow entries and code are ready to refer each other by this proposal
Next, want to observe the behavior of flows after then
•Problem : Flow entries have insufficient info to reconstruct the original path
•Reason : Wildcards and Maskbits
There are many cases which cannot figure out the correct path from flow entry info on switches
Flow merging by wildcards / maskbits
Wildcard fail case
(1.0.0.1, *, *) [#4]
(1.0.0.2, *, *) [#4]
sw1
1
2
3
4
5
6
(*, 80, *) [#4]
(*, 25, *) [#5]
sw2
1
2
3
4
5
6
to http proxy
to smtp gateway
host1
host2
convention : ( IP, src port, dst port ) [ outport ]
note:If all flow entries specify the same match fields set, it is possible to check up completely. Typical case of that is “all flows specify all match fields exactly”.
•Typical case to fail to trace the flow pathThere are wildcards in different match fields and it makes fail to reconstruct the original flow path correctly, to check up flow entries in two switches
Ex: The upper flow of sw1 (to 1.0.0.1) will be split into multi flows on sw2. (red lines)And flow entries of sw2 includes the other flows. (blue lines)Nobody knows how many packets will go upper flow of sw1 to upper flow (port 80) of sw2.
Flow tracing (how to)
•Set the label to packet and bring it by packet themselves to the next hop
•Label setting and detecting should be done by switch
It means label should be on the field of flow entry
•Set the label by “Action” then send to the next switch
•Detect the labeled flow by “Match Fields” then treat it separately from unlabeled flow
Flow separation under wildcards / maskbits
Wildcard fail case
•Separation the flow by the labelThe separated flow entry will be made in the next switch, then it is fully traceable
Ex : The case a flow will be split on the next switch. (red line)All “5” labeled packets will make only “5” matching flow entries on sw2And “5” labeled flows on sw2 does not include any unlabeled flow.
(1.0.0.1, *, *, 0) [5, #4]
(1.0.0.2, *, *, 0) [=, #4]
sw1
1
2
3
4
5
6
(*, 80, *, 5) [ =, #4]
(*, 80, *, 0) [ =, #4]
sw2
1
2
3
4
5
6
to http proxyhost1
host2to smtp gateway
(*, 25, *, 5) [ =, #5]
(*, 25, *, 0) [ =, #5]
convention : ( IP, src port, dst port, label ) [ label, outport ]
Flow tracing (the structure)
(to servers)
(to servers)
flow table
flow tableflow table
flow table
Controller
cookie Match Counter Action101, 123 IP(dst)=X out=6
cookie Match Counter Action????? ????? out=3
Packets have info for tracing and it will be a key to check up flow entries of switches
1
2
3
4
5
6
Ex : select outport by IP and the code for it
def packet_in_callback(dpid, inport, reason, len, bufid, packet): match = extract_flow(packet) iph = packet.find('ipv4') if ip_to_str(iph.dstip) == "1.0.0.1" : outport = 4 else outport = 5 actions = [[openflow.OFPAT_OUTPUT, [0, outport]]] inst.install_datapath_flow(dpid, flow,..., actions, ...) return CONTINUE
#4](1.0.0.1, *, * ) [convention : (IP, port-dst, port-src) [ #outport ]
Here is a switch that selects output port depends on the destination IP address of the packet
Flow ID Label settingdef packet_in_callback(dpid, inport, reason, len, bufid, packet): match = extract_flow(packet) iph = packet.find('ipv4') if ip_to_str(iph.dstip) == "1.0.0.1" : outport = 4 inst.markFlowLabel( 5 ) else outport = 5 actions = [[openflow.OFPAT_OUTPUT, [0, outport]]] inst.install_datapath_flow(dpid, flow,..., actions, ...) return CONTINUE
Match Fields Counters Actionsflow entry:
( label=0 ) [ set label=5 ]
, 0 5, #4](1.0.0.1, *, * ) [
Insert the function to set the Flow ID Label and the label will be set by Action feature of the switch
The place to record the Flow ID label
•Use ToS field (6bits) for IPv4
OpenFlow supports ToS as Match Field and Action
Not regularly used than other fields
• It is possible to use VLAN id or MPLS label but....
Very commonly used for OpenFlow applicable field, such as datacenter
unexpected VLAN or MPLS tag insertion may occur some malfunctions to existing controller programs
• for IPv6, Flow Label (20bits) is good to use
Label overlapping : assigning to bit
•Overlapping : set different labels to single flow by multiple switches
Cannot overwrite existent label info•Label as bit pattern
label=n means ToS=2^(n-1)independently writable for overlapping case
•Very few labels to able to set (ToS has only 6 bits!)Use string label in the code, dynamically assign the number when it turned onTurn on/off by remote operation of controller program
Restriction : Tracing flow over the switch
•Diverted field does not usable for original purpose( v4: ToS / v6: flowlabel)
•Need to remove the label on the edge
To prevent fail-reaction by the ToS value at the outside of the target Open Flow network
There are some topology investigation researches. OFDP : OpenFlow Discovery Protocol by GENI. Topology module by Trema (LLDP base)
•Not applicable for not IP protocols
For not IP protocols?
• Ideally...”Should propose to add flow-id-field as OF std.”
It's a chicken and egg situation
New spec might not applicable on existent switch
•Realistically... “not a big issue?”
OpenFlow is focusing IP (v4) strongly
In L2 use case, processed by exact match of MACs(label does not required, just checking them up is enough)
Full picture
Controller
cookie Match Counter Action
88, 101 tos=0 tos=16
markLogicPoint( 101 )markFlowLabel( 5 )
markLogicPoint( 121 )markFlowLabel( 3 )
if dpid == 25
else if dpid == 11
cookie Match Counter Action
99, 121 tos=16 tos=20
cookie Match Counter Actiontos=16 ( keep tos )
20=2^(5-1) +2^(3-1)
dpid 11
dpid 25
16=2^(5-1)
System layout
Controller
flow table
flow table
flow table
flow table
flow table aggregator
visualizer
statistics data
Controller SDK
tool
Controller App
OpenFlow standard !
operation console
labelon/off
red color means “this part should be implemented”
Conclusion
•Proposed the OpenFlow Controller development support tool•Basic design of the mechanism for binding code and flow
It enables cross referencing the logic and flow each otherIt enables tracing the flow over switches
•Required preparationNeed to add some functions to controller SDK
•To use this featureInsert the function call to your code lines, but that’s allNo special requirement for switch, just OpenFlow capability
Conclusion
•Future work• Finishing up the NOX implementation• Confirming the applicability for other SDK by porting• Visualization• Proposing a new “flow id field” as OF switch standard
operation console
flow table aggregatorvisualizer
statistics data
Controller SDKtool
Controller App
flow table
flow table
flow table
flow tableon/off
Thank you.
if you have any interest of this idea,porting to the other controller,
visualization, making a commercial product,
or joint research,
feel free to contact me please.