network monitoring - pipeline

Network MonitoringNetwork Monitoring

Review of SoftwareReview of Software

ComponentsComponentsNetwork DiscoveryNetwork DiscoveryAvailability monitoringAvailability monitoring•• Alerting systemAlerting systemService MonitoringService MonitoringNetwork PerformanceNetwork PerformanceAsset ControlAsset ControlVersion ControlVersion ControlConfiguration ManagementConfiguration ManagementHost trackingHost trackingBehaviour analysisBehaviour analysis

Network Discovery Network Discovery –– Vital FeaturesVital Features

Selectivity Selectivity •• CanCan’’t see the wood for the trees!t see the wood for the trees!

SpeedSpeed•• Network kit often in Network kit often in

huge private rangeshuge private ranges

Method of Method of automatically automatically getting results outgetting results out

Network DiscoveryNetwork Discovery

NetdiscoNetdisco•• Free network discovery package using Free network discovery package using

CDPCDP

SNMP CDP neighbour scriptSNMP CDP neighbour script

Availability MonitoringAvailability Monitoring

Scriptable configurationScriptable configurationHierarchy of the networkHierarchy of the networkNotification schedule by timeNotification schedule by timeNotification schedule by severityNotification schedule by severityNotification methodsNotification methods•• SMS, email etcSMS, email etc

Multiple viewsMultiple viewsHistorical recordHistorical record

Availability Monitoring PackagesAvailability Monitoring Packages

NagiosNagiosOpenNMSOpenNMSZabbixZabbixZenossZenossNAVNAV……etcetcMost commercial software NOT Most commercial software NOT configurable from scriptsconfigurable from scripts

Service monitoringService monitoring

Scriptable configurationScriptable configurationMore than just connect to portMore than just connect to portMany different protocolsMany different protocolsAlert scheduling by time, group and Alert scheduling by time, group and severity as availability monitoringseverity as availability monitoring

Service Monitoring PackagesService Monitoring Packages

Tests port functionTests port function•• NetcrunchNetcrunch•• OpManagerOpManager

Port up onlyPort up only•• NagiosNagios•• OpenNMSOpenNMS•• ZabbixZabbix•• ZenossZenoss

Network Performance Network Performance -- Vital Vital FeaturesFeatures

Scriptable configurationScriptable configurationTemplate systemTemplate systemScalability Scalability •• (25,000 SNMP parameters for Bangor)(25,000 SNMP parameters for Bangor)

ReliabilityReliabilityLong term supportLong term supportLong term storage Long term storage •• Data preservation across upgradesData preservation across upgrades•• Data averaging?Data averaging?

Thresholds and Automated warningsThresholds and Automated warnings

Network Performance Network Performance MeasurementMeasurement

Most MRTG inspired or Most MRTG inspired or RRDToolRRDTool based based systems sample at interval and then systems sample at interval and then progressively average.progressively average.

•• Hourly (1 minute average)Hourly (1 minute average)•• Daily (5 minute average)Daily (5 minute average)•• Weekly (30 minute average)Weekly (30 minute average)•• Monthly (2 hour average)Monthly (2 hour average)•• Yearly (1 day averageYearly (1 day average

•• Little storage required.Little storage required.•• Predefined quantity of storage required.Predefined quantity of storage required.

The effect of averagingThe effect of averaging

0

1e+08

2e+08

3e+08

4e+08

5e+08

6e+08

7e+08

8e+08

9e+08

0 20000 40000 60000 80000 100000 120000 140000

Dat

a ra

te in

Bits

/Sec

ond

(1G

b/s

= 1e

+09)

Time in Seconds

Effect of Averaging Data Rate

5 seconds30 seconds

5 minutes30 minute

2 hour1 day

Network Performance Network Performance -- StorageStorage

RRDTool

Cacti Cricket (NAV) OpenNMS Zenoss

RRDTool defaultsmimic MRTG. Most other packages takethose defaults.


MRTG style RRD AdvantagesMRTG style RRD Advantages•• Very limited storage requiredVery limited storage required•• No data growthNo data growth

No maintenance requiredNo maintenance required

MRTG style RRD DisadvantagesMRTG style RRD Disadvantages•• Useless for capacity planningUseless for capacity planning•• Rapidly loses resolutionRapidly loses resolution•• Graphs cannot be directly comparedGraphs cannot be directly compared

Network Performance Network Performance -- StorageStorageSample @ 5 Sample @ 5 secsecQuantise to Quantise to nearest % nearest % loadloadCount Count occurrences occurrences of % load for of % load for periodperiodPlot as meshPlot as mesh

Network Load - Time Series

line 1

0 0.2

0.4 0.6

0.8 1

Fraction of maximum load 0 20000

40000 60000

80000 100000

120000 140000

160000

Time in Seconds

0 20 40 60 80

100 120 140 160 180

Frequency


AdvantagesAdvantages•• Graphs comparable even when count Graphs comparable even when count

period and quantise level differentperiod and quantise level different•• Low data storage requirementsLow data storage requirements

DisadvantagesDisadvantages•• Nobody does it!Nobody does it!

Network Performance Network Performance -- StorageStorageProducts that meet requirementsProducts that meet requirements

•• StatseekerStatseekerSamples @ 1/min, averaged to 1/5min after 1 yearSamples @ 1/min, averaged to 1/5min after 1 yearVery efficientVery efficientConfigurable thresholdsConfigurable thresholds

•• RTGRTGWith sufficient effort should deliverWith sufficient effort should deliverThresholds should be quite easy to codeThresholds should be quite easy to code

•• CactiCactiTheoretically need not do MRTG averagingTheoretically need not do MRTG averagingNo thresholdsNo thresholdsFree, easy to use, produces pleasing graphsFree, easy to use, produces pleasing graphs

Asset ControlAsset Control

Automatic device trackingAutomatic device trackingPermanent link between serial Permanent link between serial number and purchasing detailsnumber and purchasing detailsAble to deal with multiple ownersAble to deal with multiple ownersAble to deal with parts of chassis Able to deal with parts of chassis switches individuallyswitches individually

Asset controlAsset control

NAVNAV•• Database design sufficientDatabase design sufficient•• Front end more limited?Front end more limited?

OpenNMSOpenNMS•• Database design seems inadequateDatabase design seems inadequate

Version ControlVersion Control

Scriptable configurationScriptable configurationAutomaticAutomaticEasy identification of devices needing Easy identification of devices needing upgradeupgradeVersion historyVersion history

Configuration ManagementConfiguration Management

Scriptable configurationScriptable configurationSave running configurationSave running configurationRun scripted commandsRun scripted commandsAlert on configuration changeAlert on configuration changeSave configuration historySave configuration historyAlert on improperly configured Alert on improperly configured devicesdevices•• Use templates & central Use templates & central configconfig

generationgeneration

Configuration ManagementConfiguration Management

RANCIDRANCID•• Uses CVS or subversionUses CVS or subversion•• ConfigConfig backup and change detectionbackup and change detection•• No No templatingtemplating

CheetahCheetah•• TemplatingTemplating softwaresoftware

Host TrackingHost Tracking

Scriptable configurationScriptable configurationLocate host by IP or MAC addressLocate host by IP or MAC addressLocation historyLocation history

Host trackingHost tracking

With historical recordWith historical record•• NAVNAV

On the fly onlyOn the fly only•• NetdiscoNetdisco•• NetcrunchNetcrunch

Behaviour AnalysisBehaviour Analysis

Mirroring portMirroring portUsing Using netflownetflow or or sflowsflow data data

Mirroring methods need lots of CPUMirroring methods need lots of CPUAimsAims•• IDSIDS•• User behaviour analysis and controlUser behaviour analysis and control•• Improving efficiency on expensive linksImproving efficiency on expensive links

Behaviour AnalysisBehaviour Analysis

Specialist packagesSpecialist packagesSnortSnort mirroringmirroring freefreeInMonInMon sflowsflow commercialcommercialArgusArgus netflow/sflow/mirroringnetflow/sflow/mirroring

freefreeNtopNtop mirroringmirroring freefree

CactiCacti

•RRDTool based

•Graphing package

•CLI and API (API docs?)

•Scalable

Cacti featuresCacti features

Graph HierarchyGraph Hierarchy•• Difficult to configure from the commandDifficult to configure from the command

lineline

User managementUser managementAny Any OIDsOIDsGraphical managementGraphical managementEfficient poll Efficient poll Free!Free!

RDTRDT

Fast SNMP data collectionFast SNMP data collectionStorage in SQL databaseStorage in SQL databaseNot really a complete solution even Not really a complete solution even for performance monitoring for performance monitoring –– but a but a good basis?good basis?

StatseekerStatseekerNetwork performanceNetwork performanceSome availability functionsSome availability functionsNot free Not free Scales easily to University size networksScales easily to University size networksOnly software to meet most of our Only software to meet most of our network performance specificationnetwork performance specificationUsed by many Universities Used by many Universities –– including us!including us!Highly recommendedHighly recommendedVersion 3 now outVersion 3 now out

NetcrunchNetcrunch

Service based Service based –– lots of predefined serviceslots of predefined services•• Intelligent ping Intelligent ping –– not just port innot just port in

Logical map Logical map –– graphical mapping graphical mapping Manual physical viewsManual physical viewsPerformance monitoring can be configured Performance monitoring can be configured -- not designed to be run by default on all not designed to be run by default on all ports.ports.Servers performance monitoring through Servers performance monitoring through SNMPSNMP

NetCrunchNetCrunchCannot do everything from web, but all Cannot do everything from web, but all monitoring except trafficmonitoring except traffic22--3 days consultancy on installation3 days consultancy on installationXE XE –– unlimited unlimited ££11,500 11,500 –– down to down to ££4,000 4,000 smallest limited versionsmallest limited version•• ££3,690 software maintenance 3,690 software maintenance –– major and major and

minor updates & telephone supportminor updates & telephone support

Central behaviour analysis solution Central behaviour analysis solution --NetfortNetfort•• ££25,00025,000

NAVNAV

Database centred frameworkDatabase centred frameworkUsed by all Norwegian UniversitiesUsed by all Norwegian UniversitiesGood documentationGood documentation•• Database design documented etcDatabase design documented etc

Designed for UniversitiesDesigned for UniversitiesFreeFree

NAVNAV

Availability monitoringAvailability monitoring•• Alerting systemAlerting system

Service Monitoring?Service Monitoring?Network Performance Network Performance -- cricketcricketAsset Control Asset Control Version ControlVersion ControlHost trackingHost trackingWeathermapWeathermap

NAVNAV

ButBut……Performance monitoring is poor Performance monitoring is poor (cricket)(cricket)Can be difficult to install Can be difficult to install –– best on best on debiandebianMailing list traffic lowMailing list traffic low

Open NMSOpen NMS

Service monitoringService monitoringAvailability monitoringAvailability monitoringPerformance monitoringPerformance monitoringSome asset trackingSome asset tracking

Buggy and unpredictableBuggy and unpredictablePrimarily service monitoring Primarily service monitoring –– rest rest seems to be an afterthoughtseems to be an afterthought

ZabbixZabbix

AvailabilityAvailabilityPerformance monitoringPerformance monitoringService monitoringService monitoring

Difficult to get startedDifficult to get startedDoes not seem to excel at anythingDoes not seem to excel at anything

ZenossZenoss

Commercial backedCommercial backedService monitoringService monitoringAvailability monitoringAvailability monitoringSome performanceSome performance

Buggy and erraticBuggy and erratic

OpManagerOpManager

Availability monitoringAvailability monitoringAdvanced service monitoringAdvanced service monitoringCheapish commercial productCheapish commercial product

Asset control extraAsset control extraPoor performance monitoringPoor performance monitoring

NTopNTop

FreeFreeNetwork monitoring behaviour Network monitoring behaviour analysisanalysisEasy to installEasy to installPretty graphical outputPretty graphical outputEasy to understandEasy to understand

Availab

ility Availab

ility M

onito

ring

Monito

ring

Service

Service

Monito

ring

Monito

ring

Netw

ork

Netw

ork

Perform

ance

Perform

ance

Asset C

ontro

lAsset C

ontro

l

Versio

n C

ontro

lVersio

n C

ontro

l

Config

uratio

n

Config

uratio

n

man

agem

ent

man

agem

ent

Host T

racking

Host T

racking

Netw

ork

Netw

ork

Disco

veryD

iscovery

Beh

aviour

Beh

aviour

Analysis

Analysis

CactiCacti

ZabbixZabbix

Open Open NMSNMS

NAVNAV??

RancidRancid

SnortSnort

Bangor UniversityBangor University’’s Choicess Choices

NetdiscoNetdisco or inor in--house SNMP CDP house SNMP CDP scriptscript•• Network discoveryNetwork discovery

NagiosNagios•• AvailabilityAvailability•• Service monitoringService monitoring

StatseekerStatseeker•• Network performanceNetwork performance

Bangor UniversityBangor University’’s Choicess ChoicesNAVNAV•• Core database for network managementCore database for network management•• Asset controlAsset control•• Host trackingHost tracking•• Availability?Availability?•• Version control?Version control?

RANCIDRANCID•• Configuration managementConfiguration management

CheetahCheetah•• Configuration Configuration templatingtemplating

Snort/Snort/NtopNtop•• Behaviour analysisBehaviour analysis

Any questions or observationsAny questions or observations

??????????

network monitoring - pipeline

Documents