network debugger: a unified tool for diagnosing network …wtc2012/slides/workshops/ws2-1/ws... ·...
TRANSCRIPT
Network Debugger:A Unified Tool for Diagnosing Network
Controlling Applications
WTC 2012 Workshop on Software Defined Networks (SDN) and OpenFlow
March 7, 2012Yasunobu Chiba and Hideyuki Shimonishi
System Platforms Research LaboratoriesNEC Corporation
Outline
▐ Background and problem statement▐ High-level design of Network Debugger▐ Preliminary design and implementation results▐ Conclusion and future work
© NEC Corporation 2012Page 2
Background
▐ Software-Defined Network (SDN)
•Network elements are provided by hardware vendors and they are in
Current
•Features provided in networks can be flexibly defined or updated
SDN Era
vendors and they are in operation for 5-10 years
•Features cannot be added while users require new features or have new ideas
defined or updated
•Features can be added by users (not hardware vendors)
•New features can be easily evaluated and chosen for production use
© NEC Corporation 2012Page 3
Background – cont’d
▐ Open Networking Foundation says…
▐ Are you sure about its success?
https://www.opennetworking.org/media-room/press-releases/63-open-networking-foundation-formed-to-speed-network-innovation
SDN thus gives owners and operators of networks better control over their networks, allowing them to optimize network behavior to
best serve their and their customers’ needs.
at this moment…
© NEC Corporation 2012Page 4
Issues
▐ No sophisticated debugging environment for developing control plane application is provided
▐ Current/legacy networks are standards-based and they have been implemented/deployed by spending many years
▐ In SDN, various non-standards-based features are implemented/deployed in a timely manner based on user’s implemented/deployed in a timely manner based on user’s requirements� It is hard to develop purpose-built validation/diagnosis tools for each
specific feature due to short development period and cost constraint� If we develop purpose-built tools, it reduces flexibility, a notable
benefit of SDN
Providing a unified tool for diagnosing control plane applications is a key success factor of SDN
© NEC Corporation 2012Page 5
Goal and scope of work
▐ Provide Network Debugger – A unified tool for diagnosing various control plane applications
Control Plane Application / OpenFlow Controller
Software-Defined / OpenFlow Network
Module A Module B Module Z
Controller Platform
State / Event
Collection
State / Event
Analysis
Scope of Network DebuggerScope of rest of presentation
Switch
Host
Emulated / Real Network
Network Debugger Console
(UI)
Breaking
Single-stepping
Network Debugger provides similar functionality to debugger for standalone application software
© NEC Corporation 2012Page 6
Collecting states/events from network: Past and current
Log
LogLog
Log
Log
Log
Log
Log
DUT/SUT
Collect logs
•Collect states/events on network elements using various ways and then merge and serialize them at a certain place
•See if a network is in proper state by comparing the state/event series with an expectation constructed from product and protocol knowledge
•Multiple mechanisms depending on network elements are required for collecting states/events
Combine logs into a single event series
Sent packet A to …0.00Sent packet B to …0.12Sent packet A to …0.15Validate packet A …0.29Sent packet B to …0.52Receive packet A …0.81
Find out focusing events
Sent packet A to …0.00Sent packet A to …0.15Validate packet A …0.29Receive packet A …0.81
Sent packet A to …Sent packet A to …Validate packet A …Receive packet A …
Log
LogLog
Log Log
Log
Log
Log
DUT/SUT
Compare with expectation
Protocol Specification
Product Manual
collecting states/events
•Some state/event collection mechanisms seriously interfere in operation status (heavily consume CPU/memory resource etc.)
•Real-time state/event collection is hard
© NEC Corporation 2012Page 7
Collecting states/events from network: Proposed
State / Event Notifications w/
Control Plane Application /
OpenFlow Controller
Developer / Operator
Real-time / Off-line Monitor
•Centrally collect states/events from various network elements with a common format and a light-weight transport in real-time or offline
•Provide a tool (state/event viewer) to analyze a huge amount of collected states/events efficiently/easily
State / Event Viewer
File
Event Collector
Circular Buffer
State / Event Notifications w/ Common Notification Format
Network Interface / Tap
Switch
Host
Serialized State / Event Notifications
© NEC Corporation 2012Page 8
Effectiveness and issue
▐ Effectiveness� All states/events can be collected at a single place in real-time with a
common light-weight collection mechanism� Application developers/operators can focus on analysis of collected
states/events� State/event filter functions embedded in the state/event viewer helps
to find a focusing event series easily
▐ Issue� Finding a context (a series of events that need to be addressed)
should still be done with experienced application developers/operators
© NEC Corporation 2012Page 9
Proof-of-concept implementation: Design
Event
Notifications through Trema IPC (messenger)
Switch
Trema-based OpenFlowController
Host
Developer / Operator
Real-time / Off-line Monitor
SyslogRelay
Syslog
State / Event Viewer
IPC mechanism provided by Trema is reused as a light-weight
transport for our method
State/event notification mechanism is
implemented in Trema
Wireshark with a plug-in for our method is used as state/event viewer
Event Collector
Circular Buffer
Network Interface / Tap
Serialized Notifications
Relay
Packet Capture
StdinRelay
PcapFile
Wireshark w/ plugin
Any Text String
Bridges are provided for accommodating legacy state/event
collection mechanisms© NEC Corporation 2012Page 10
Proof-of-concept implementation: State/Event Viewer
Inter-function module events
Packet capture
Syslog message
Packet capture
OpenFlow Controller
learning_switch
switch.abc
27. Packet-In
28. Packet-In29. Packet-In
31. Packet-Out30. Packet-Out
32. Packet-Out
34. Error !!!
© NEC Corporation 2012Page 11
Evaluation of PoC implementation: Metrics
1. State/event collation performance� Measure events per second performance with our method and a
legacy event collection mechanism (i.e. syslog)
2. CPU usage� Assume a case that software developer or operator confirms that all
flow entries are properly installed from an OpenFlow controller into flow entries are properly installed from an OpenFlow controller into OpenFlow switches in an OpenFlow network
� Measure CPU usage in operation with our method and a legacy event collection mechanism (i.e. syslog)
© NEC Corporation 2012Page 12
State/event collation performance: Result
100,000
150,000
200,000
# o
f E
ve
nts
Co
llecte
d [e
ve
nt/
se
c]Event Collection Performance
Proposed Method Syslog (rsyslog) Syslog (syslog-ng)
Our method collects states/events with higher rate than syslog implementations
0
50,000
# o
f E
ve
nts
Co
llecte
d [e
ve
nt/
se
c]
# of Events Generated [event/sec]
© NEC Corporation 2012Page 13
CPU usage: Result
40
50
60
70
CP
U U
sage [%
]CPU Usage in Operation
W/o Event Collection Proposed Method Syslog (rsyslog) Syslog (syslog-ng)
Our method collects states/events with lower side effects than syslogimplementations in terms of operation status
0
10
20
30
100 300 500 700 900
CP
U U
sage [%
]
Flow Setup Rate [flow/sec]
© NEC Corporation 2012Page 14
Conclusion and future work
▐ Conclusion� Presented a vision for Network Debugger which should be a key
success factor of SDN� Proposed and implemented a mechanism for collecting states/events
from various network elements including control plane application, switches/routers, and etc. with a common and light-weight way
� Confirmed that our method is able to collect events with higher rate and lower CPU utilization than a legacy event collection mechanismand lower CPU utilization than a legacy event collection mechanism
▐ Future work� Usability test in real software development/operation cases� Integration with a network emulator� Design and implementation of a feedback mechanism from console UI
to emulated network
© NEC Corporation 2012Page 15
Conclusion and future work – cont’d
▐ Some parts of this work are shipped with Trema OpenFlowController Platform� http://trema.github.com/github� https://github.com/trema/trema
© NEC Corporation 2012Page 16
NEC Group Vision 2017
To be a leading global companyleveraging the power of innovation
to realize an information societyfriendly to humans and the earthfriendly to humans and the earth