an analysis of network configuration artifacts · 2019. 2. 25. · motivation and goals like...
TRANSCRIPT
![Page 1: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/1.jpg)
An Analysis ofAn Analysis ofNetwork Configuration ArtifactsNetwork Configuration Artifacts
David Plonka & Andres Jaan Tack{plonka,tack}@cs.wisc.edu
LISA '09, November 5, 2009
![Page 2: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/2.jpg)
Motivation and Goals● Like software quality, network reliability is evolving:
– Expectation of high availability, increasing reliance
– Increasing numbers of skilled practitioners
– Increasing level of automation
![Page 3: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/3.jpg)
Motivation and Goals● Like software quality, network reliability is evolving:
– Expectation of high availability, increasing reliance
– Increasing numbers of skilled practitioners
– Increasing level of automation
● However, the management of networks and the Internet has not received similar attention to the development of software.
![Page 4: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/4.jpg)
Motivation and Goals● Like software quality, network reliability is evolving:
– Expectation of high availability, increasing reliance
– Increasing numbers of skilled practitioners
– Increasing level of automation
● However, the management of networks and the Internet has not received similar attention to the development of software.
● We propose an analogy-based analysis, and that these elements are akin to each other:– Networks : Software Systems
– Network Engineering : Software Engineering
– Network Operators : Programmers
![Page 5: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/5.jpg)
Campus Network
![Page 6: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/6.jpg)
Network Artifacts● artifact - an object created by humans,
especially one remaining from a particular period● Network Performance Measurements● Network Management Systems' Topology● Trouble Tickets● Network Device Configurations
– Routers, switches, firewalls
– Network practitioners use Source Code Management (SCM) of device configurations for:
– Configuration backups
– Communicating changes
![Page 7: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/7.jpg)
Network Configuration Repositories
![Page 8: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/8.jpg)
Networks Studied
![Page 9: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/9.jpg)
Mining SCM Repositories - Why?● While successful in the PL community, this
hasn't been leveraged in the context of network configuration and management.
● To visualize and elucidate network operation with the goal of understanding and improving the practice.
![Page 10: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/10.jpg)
Mining SCM Repositories - How?● Convert existing custom network version
control system repositories to common CVS repositories.
● Use existing tools from the Programming Language (PL) and open source developer communities, e.g.:– StatCVS-XML
– cvs2cl (CVS to ChangeLog)
● Perform additional static file analyses, e.g.:– Syntax-aware statistics (i.e. config stanzas)
– Revision lifetimes
![Page 11: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/11.jpg)
Configuration Files / Code Sampleversion 12.2
no service pad
service timestamps debug datetime localtime
service timestamps log datetime localtime
service password-encryption
!
hostname s-bldg-5-2-access
!
spanning-tree mode rapid-pvst
no spanning-tree optimize bpdu transmission
spanning-tree extend system-id
!
![Page 12: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/12.jpg)
Code Sample (2)interface FastEthernet1/0/1
description sample 100Mbps ethernet interface
switchport access vlan 42
switchport mode access
ip access-group nodhcpserver in
snmp trap mac-notification change added
snmp trap mac-notification change removed
no snmp trap link-status
no mdix auto
spanning-tree portfast
spanning-tree bpduguard enable
spanning-tree guard root
!
![Page 13: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/13.jpg)
Code Sample (3)ip access-list extended nodhcpserver
remark Id: ndhcp.acl,v 1.2 2005-05-20 11:26:03 ashley Exp
deny udp any eq bootps any
permit ip any any
!
access-list 5 permit 192.2.0.1
access-list 5 remark Allow foo, bar, and baz servers
access-list 5 permit 192.2.0.10
access-list 5 permit 192.2.0.11
!
...
!
end
![Page 14: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/14.jpg)
Campus File / Device Count
![Page 15: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/15.jpg)
Campus LOC by Topology
![Page 16: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/16.jpg)
Campus LOC per Module
![Page 17: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/17.jpg)
Campus Size Per Author
![Page 18: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/18.jpg)
Campus Size Per Group
![Page 19: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/19.jpg)
Campus Commits by Hour
![Page 20: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/20.jpg)
Common Commit Comments
![Page 21: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/21.jpg)
An Anomaly
![Page 22: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/22.jpg)
Web-based Network Administration
![Page 23: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/23.jpg)
● Measurements of practitioner effort– How often are “fixes” introduced?
– How often do configurations change?
– “Bad Days” (are Friday checkins more buggy?)
● Look toward improvements:– Syntax-aware revision analysis (stanzas)
– How do we direct tool development?
Evaluating Practitioner Effort
![Page 24: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/24.jpg)
Campus Commits by Day
![Page 25: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/25.jpg)
● How long does a revision last before it is next modified?– Suggests the modus operandi of practitioners
– Suggests the value or the staying power of a revision
– Might also suggest some measure of network volatility
Revision Lifetimes
![Page 26: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/26.jpg)
Campus Revision Lifetimes (<3.5 days)
![Page 27: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/27.jpg)
Campus Revision Lifetimes (<10 min)
![Page 28: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/28.jpg)
% Short-Lived Revisions by Day
Sun Mon Tue Wed Thu Fri Sat
0
1
2
3
4
5
6
7
8
Service ProviderCampus
![Page 29: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/29.jpg)
% Short-Lived Revisions by Day
Sun Mon Tue Wed Thu Fri Sat
0
5
10
15
20
25
agentscontractfieldnetnoc
(Campus Drill-Down)
![Page 30: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/30.jpg)
Campus Average File Size
![Page 31: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/31.jpg)
Service Provider Average File Size
![Page 32: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/32.jpg)
Campus Revisions by Stanza Type
![Page 33: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/33.jpg)
Some Conclusions● With varying device types, LOC is an erratic
metric for the stanza-based, declarative network configuration language, (such as Cisco IOS)
● Analysis of network configurations exposes pertinent network management details including:– Group behaviors
– Outstanding practitioners
– Change times
– High level of user compliance, but some curiosities
– Tool-based efficiencies both expected and invented
![Page 34: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/34.jpg)
Contributions● An initial application of software development
analysis tools to network operations based on existing, freely-available tools
● Beginnings of a network operations-specific measurement of practitioner effort to guide tool development, such as SCM and IDE-like tools for network operators
● In our case studies, this analogy-based analysis approach shows promise based on feedback by expert interviews.
![Page 35: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/35.jpg)
Discussion and Future Work
● As in software, can we identify and investigate code decay, refactorings, and code clones?
● Leverage other artifacts to measure practitioner compliance and network service reliability and performance.
● Develop a complexity metric based on stanzas and inter-stanza references. (see Benson, et al., NSDI 2009)
![Page 36: An Analysis of Network Configuration Artifacts · 2019. 2. 25. · Motivation and Goals Like software quality, network reliability is evolving: – Expectation of high availability,](https://reader033.vdocument.in/reader033/viewer/2022061005/60b3140329fe4851fd15183e/html5/thumbnails/36.jpg)
An Analysis ofAn Analysis ofNetwork Configuration ArtifactsNetwork Configuration Artifacts
David Plonka & Andres Jaan Tack{plonka,tack}@cs.wisc.edu
LISA '09, November 5, 2009