solace integration with nagios monitoring toolset...solace nagios toolset integration 8 3.3 post...

65
Copyright © Solace Systems, Inc. http://www.solacesystems.com Solace Integration with Nagios Monitoring toolset Document Version 0.5 December 2015 This documents is an integration guide for monitoring Solace Message Routers with Nagios and related tools such as Nagiograph and RRDTool. Nagios is an open source monitoring system widely deployed to monitor IT infrastructure including computer systems, network, applications and services. Solace message routers unify many kinds of data movement so companies can efficiently and cost-effectively move all of the information associated with better serving customers and making smarter decisions. The Solace 3560 message router is the highest performance data movement technology available, with the capacity and robustness to support the most demanding enterprise messaging, big data, cloud computing and Internet of Things applications.

Upload: others

Post on 16-Apr-2020

26 views

Category:

Documents


0 download

TRANSCRIPT

Copyright copy Solace Systems Inc

httpwwwsolacesystemscom

Solace Integration with

Nagios Monitoring toolset Document Version 05

December 2015

This documents is an integration guide for monitoring Solace Message Routers with Nagios and

related tools such as Nagiograph and RRDTool

Nagios is an open source monitoring system widely deployed to monitor IT infrastructure including

computer systems network applications and services

Solace message routers unify many kinds of data movement so companies can efficiently and

cost-effectively move all of the information associated with better serving customers and making

smarter decisions The Solace 3560 message router is the highest performance data movement

technology available with the capacity and robustness to support the most demanding enterprise

messaging big data cloud computing and Internet of Things applications

Solace Nagios Toolset Integration

2

Table of Contents

Contents Table of Contents 2 1 Overview 4

11 Audience 5 12 Related Documentation 5 13 Prerequisites 5

131 Requirements 5 132 Assumptions 5

2 Why Solace 6 Superior Performance 6 Robustness 6 Simple Architecture 6 Simple Operations 6 Cost Savings 6

3 Nagios Setup 7 31 Prerequisites 7 32 Installation 7 33 Post install setup 8

331 HTTP Setup 8 332 Firewall setup 8 333 Nagios Admin User 8

34 Verification 9 4 Nagiosgraph Setup 10

41 Prerequisites 10 42 Installation 10 43 Post install setup 11

431 Restarting Services 11 44 Verification 12

5 Solace Nagios Plugin Setup 14 51 Prerequisites 14 52 Installation 14 53 Post Install Setup 16 54 Verification 16

6 NSCA Setup 18 61 Prerequisites 18 62 Installation 18 63 Post install setup 19 64 Verification 20

641 Sending Host Alert 20 642 Sending Service Alert 21

7 NRPE Setup 22 71 Prerequisites 22 72 Installation 22

721 Server (CentOS) 22 722 Client (Ubuntu) 22

73 Post Install Configuration 23 731 Client Configuration 23 732 Server Configuration 23

74 Verification 24 75 NCSA vs NRPE 24

8 Cacti Setup 25 81 Prerequisites 25

811 Cacti Dependencies 25 82 Installation 25 83 Post Install Setup 25

831 Add Custom Data Input method 26 832 Add Custom Data Template 27 833 Add a custom Graph Template 28

Solace Nagios Toolset Integration

3

834 Add Solace Appliance as Device 29 84 Verification 31

841 Basic Install 31 842 Solace host check 32 843 Solace VPN Metrics Sample 32

9 Additional Topics 34 91 Admin Tasks 34

911 Creating custom Maps 34 912 Adding additional routers VPNs 34 913 Putting Nagios on Verbose mode 34 914 Modifying iptables 35

92 Troubleshooting 36 921 Nagios not starting up 36 922 Canrsquot connect to HTTP (or other port) from outside 36 923 CGI Error in showcgi 36 924 CGI Error in Solace graphs 37 925 Canrsquot stat command file error 37

10 Setup Scripts and Config files 38 101 Install Info 38 102 Nagios 39

1021 setup_nagiossolace 39 1022 mk_nagiossolacecfg 45 1023 Solace Nagios Config (cfgsolacecfg) 47 1024 Template router (cfgsolace-templateroutercfg) 50 1025 Template and Sample router Config 53

103 Solace Nagios Plugin 53 104 NSCA 54

1041 setup_nsca 54 1042 send_ngalertpl 57 1043 nsca_init 61 1044 NSCA Config 63

105 NRPE 64 1051 Sample server config file 64 1052 Sample Client Config 65

Solace Nagios Toolset Integration

4

1 Overview Nagios (previously NetSaint) is an open-source monitoring and alerting system thatrsquos widely used to monitor systems

networks and infrastructure Objects monitored by Nagios are split into two categories hosts (physical machines) and

services (particular functionalities) Nagios does not perform any host or service checks on its own and relies on plugins

to do this This makes it a very modular and flexible solution

Extending this ldquomonitoring by pluginrdquo paradigm of Nagios Solace provides a set of plugin scripts that perform the

Appliance and VPN monitoring Following Nagios convention the plugins add host (router) and service (VPN

resources) categories

Nagiograph parses output and performance data from Nagios plugins and generates graphs and HTML pages for

reporting Nagiograph stores data in RRD (Round Robin Database) files as time series data RRDTool is an Open

source parser that integrates into Shell scripts Perl Python Ruby Tcl etc

The following picture illustrates different components and high level data flow

Solace Nagios Toolset Integration

5

11 Audience This document is primarily intended for Solace administrators and operations teams responsible for setting up and

monitoring infrastructure

12 Related Documentation These documents contain information related to the feature defined in this document

Document ID Document Title Document Source

[Solace-Portal] Solace Developer Portal httpdevsolacesystemscom

[Solace-FG] Solace Messaging Platform ndash Feature

Guide

httpdevsolacesystemscomdocsmessaging-

platform-feature-guide

[Solace-FP] Solace Messaging Platform ndash Feature

Provisioning

httpdevsolacesystemscomdocsmessaging-

platform-feature-provisioning

[Solace-CLI] Solace Message Router Command Line

Interface Reference

httpdevsolacesystemscomdocscli-reference

Table 1 - Related Documents

13 Prerequisites

131 Requirements

The installation and setup steps require that you have root access on the CentOS server

The installation and setup steps also require that you have CLI access to the Solace appliance

The Solace appliance should be reachable over network from the CentOS server running Nagios

132 Assumptions

This document is based on SolOS version 711 though earier versions (such as SolOS 62) should work

without any changes

Setup steps in this guide were tested on CentOS 63 CentOS 70 (both 64 Bit Minimal install) Command

syntax and file locations would vary on other flavors of Linux

Solace Nagios Toolset Integration

6

2 Why Solace Solace technology efficiently moves information between all kinds of applications users and devices anywhere in the

world over all kinds of networks Solace makes its state-of-the-art data movement capabilities available via hardware

and software ldquomessage routersrdquo that can meet the needs of any application or deployment environment Solacersquos

unique solution offers unmatched capacity performance robustness and TCO so our customers can focus on seizing

business opportunities instead of building and maintaining complex data distribution infrastructure

Superior Performance Solacersquos hardware and software messaging middleware products can cost-effectively meet the performance needs of

any application with feature parity and interoperability that lets companies start small and scale to support higher

volume or more demanding requirements over time and purpose-built appliances that offer 50-100x higher

performance than any other technology for customers or applications that require extremely high capacity or low

latency

Robustness Solace offers high availability (HA) and disaster recovery (DR) without the need for 3rd party products and fast failover

times no other solution can match Distributing data via dedicated TCP connections ensures an orderly well-behaved

system under load and patented techniques ensure that the performance of publishers and high-speed consumers is

never impacted by slow consumers

Simple Architecture Modern enterprises run applications that demand many kinds of data movement such as persistent messaging web

streaming WAN distribution and cloud-based communications By supporting all kinds of data movement with a unified

platform that can be deployed as a small-footprint software broker or high-capacity rack-mounted appliance Solace lets

architects design an end-to-end infrastructure thatrsquos easy to build applications for integrate with existing technologies

secure and scale

Simple Operations Solacersquos solution features a shared administration framework for all kinds of data movement deployment models and

network environments so itrsquos easy for IT staff to deploy monitor manage and upgrade their Solace-based messaging

environment

Cost Savings Solace reduces expenses with high-capacity hardware flexible software and the ability to deploy the right solution for

each problem Solacersquos support for many kinds of messaging lets you replace multiple messaging products with just

one built-in HA DR WAN and Web functionality eliminate the need for third-party products

Solace Nagios Toolset Integration

7

3 Nagios Setup

31 Prerequisites A working Nagios installatin requires the following packagestools

Httpd (Apache)

RRDTool

GD Grahics library and related Perl modules

NRPE (Nagios Remote Plugin Executor)

yum install -y httpd php gcc glibc glibc-common gd gd-devel make net-snmp wget

yum install rrdtool perl-rrdtool perl-GD

yum install perl-CGI perl-Time-HiRes

1 Enabling EPEL

The steps for enabling Extra Packge Repositor (EPEL) are release specific and please check for your install

CentOS 7

wget httpdlfedoraprojectorgpubepel7x86_64eepel-release-7-5noarchrpm

rpm -ivh epel-release-7-5noarchrpm

CentOS 63

rpm -Uvh httpdlfedoraprojectorgpubepel6x86_64epel-release-6-8noarchrpm

rpm -Uvh httprpmsfamillecolletcomenterpriseremi-release-6rpm

32 Installation

yum -y install nagios nagios-plugins-all nagios-plugins-nrpe nrpe

Solace Nagios Toolset Integration

8

33 Post install setup

331 HTTP Setup

Setting up an HTTP server (such as Apache) is beyond the scope of this document Pl refer to the documentation for

your HTTP server Nagios setup should have dropped the required config file to setup URL Aliases The file is typically

ltOptioalConfigDirgtnagioscfg (Eg etchttpdconfdnagioscfg)

If Checking Webadmin below doesnrsquot work check the httpd config and logs to see if this file was processed

332 Firewall setup

Firewall setup and making changes to existing firewall configuration is again beyond the scope of this document For

eg if you are using iptables (default firewall on CentOS) then you may need to explicitly allow incoming connection on

required ports (Refer Additional Topics Section)

333 Nagios Admin User

Create password for nagiosadmin user for web access

htpasswd ndashc etcnagiospasswd nagiosadmin

systemctl restart httpd

systemctl restart nagios

Solace Nagios Toolset Integration

9

34 Verification Point the webbrowser to httpyour-server-ip[port]nagios

Login with the username nagiosadmin and the password above step (htpassword command)

Sample page is shown below

Solace Nagios Toolset Integration

10

4 Nagiosgraph Setup

41 Prerequisites NagiosGraph installation requires the following packagestools

RRDTool and related Perl modules

GD Graphics Library and related Perl modules

yum install rrdtool perl-rrdtool perl-GD

42 Installation

cd optpkg or path of your choice

wget httpdownloadssourceforgenetprojectnagiosgraphnagiosgraph152nagiosgraph-

152targzwget

tar xvzf nagiosgraph-152targz

cd nagiosgraph-152

installpl --check-prereq

hellip

installpl --layout standalone --prefix usrlocalnagiosgraph --nagios-perfdata-

file=varlognagiosperfdatalog --var-dir=varnagios

hellip

hellip

Continue with this configuration [y] y

hellip

Solace Nagios Toolset Integration

11

43 Post install setup Run the Solace provided Nagios that modify the Config files appropriately Edit the script and change the file paths if

necessary at the top of the script

Running setup_nagiossolace can be deferred until Solace plugins are installed (in following section)If Solace specific

tools are not in place or the Config files not setup corresponding steps will be skipped

431 Restarting Services

chkconfig httpd on ampamp chkconfig nagios on

systemctl restart httpd

systemctl restart nagios

setup_nagiossolace

Checking required files

hellip

Setting up config files

hellip

Restarting services

nagios

nagios (pid 4155) is running

httpd

hellip

Solace Nagios Toolset Integration

12

44 Verification The above script adds another host by name ldquolocalhost-grrdquo with graphics enabled that can be used to test the

Nagiosgraph install Sample page is shown below

Solace Nagios Toolset Integration

13

Clicking on the graph symbols to the right of service name brings up a graph with various time bracket A sample graph

for one of the services (HTTP) for a day is shown below

Solace Nagios Toolset Integration

14

5 Solace Nagios Plugin Setup

51 Prerequisites Solace Nagios plugin requirs the following tools packages

Perl modules to handle XML amp Nagios

Solace pluging package

yum install perl-CPAN perl-XML-LibXML perl-Nagios-Plugin

perl -MCPAN -e shell

cpangt install NagiosPlugin

cpangt install NagiosPluginDieNicely

cpangt install LWPUserAgent

cpangt install NagiosConfig

cpangt exit

52 Installation

1 Create Sample file

Create a sample router Config This file used by the setup script (below) to create commands for this config This step

can also be repeated for other routersvpns using the command mk_nagiossolacecfg script

cat cfgsolace-vmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=19216856102

HOSTPORT=8080

USERNAME=admin

PASSWORD=

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

Solace Nagios Toolset Integration

15

2 Setup Plugin

Run the setup_nagiossolace script This will setup both Nagiosgraph and the solace plugin If Nagisgraph is already

setup the related steps will be skipped

mk_nagiossolacecfg cfgsolace-vmr2cfg

Generating Nagios config for Solace appliance

Using cfgfile cfgsolace-vmr2cfg

Using template file cfgsolace-templaterouter-no_bi_bridgecfg

Generating config

etcnagiosobjectssolace-solace-vmr2cfg

Setting up config files

etcnagiosnagioscfg

Solace Nagios Toolset Integration

16

53 Post Install Setup

54 Verification If the sample router info is successfully configured a host entry matching the appliance hostname from Config file will

be added to the web page

A sample entry is shown below

Solace Nagios Toolset Integration

17

Clicking on the graph icon to the right of the service name brings up the graphs with options A sample graph is shown

below

Solace Nagios Toolset Integration

18

6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the

external command file Nagios would periodically check the external command file and process them This ldquopassive

checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured

hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status

updates and alerting

61 Prerequisites

62 Installation

1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291

(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)

2 Untar and follow install instructions in the package

$ cd optpkg (or directory of your choice)

$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site

for latest package name and replace)

$ tar xvzf nsca-291targz (adjust version accordingly)

$ cd optpkgnsca-291 (adjust version accordingly)

$ configure

Configuration summary for nsca 291 01-27-2012

General Options

-------------------------

NSCA port 5667

NSCA user nagios

NSCA group nagios

$ make all

Solace Nagios Toolset Integration

19

63 Post install setup

1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca

This will generate required Config file and setup nsca as service

NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP

setup_nsca -s optpkgnsca-291

srcdir optpkgnsca-291 ok

target dir usrlocalnsca ok

Checking

hellip

Installing

hellip

Creating service

Installation complete

2 Start service

Now that the component is installed start it up

systemctl restart nsca

Solace Nagios Toolset Integration

20

64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl

ndashh for usage info

641 Sending Host Alert

send_ngalertpl -n vmr3 -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending host command vmr30 - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0

Output - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe

[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection

Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -

[Status=UP]

Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]

Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]

Solace Nagios Toolset Integration

21

642 Sending Service Alert

send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending service command vmr3hardware0 - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service

Description hardware Return Code 0 Output - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe

Nov 9 154116 centos7-vm nsca[9888] End of connection

Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND

PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]

Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]

Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]

Solace Nagios Toolset Integration

22

7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing

monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an

active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be

configured on both the server and the client end though the client end changes are relatively light

In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server

as client

71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that

network connectivity exists between the server and client hosts and the ports are open both ways

72 Installation

721 Server (CentOS)

In addition to Nagios server packages we would need to install the following These were already covered in previous

sections and listed here for completeness and for any standalone installs

NRPE

Nagios Pluggins

OpenSSL

yum install -y nrpe nagios-plugins-all openssl

722 Client (Ubuntu)

On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component

apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-

standard

Solace Nagios Toolset Integration

23

73 Post Install Configuration

731 Client Configuration

1 Add Nagios server IP to allowed_hosts in nrpecfg

grep ^allowed_hosts etcnagiosnrpecfg

allowed_hosts=127001192168562011921681223

2 Restart service

systemctl start nagios-nrpe-server

732 Server Configuration

1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in

ldquoSetup Scripts and Config Filesrdquo section

2 Update nagioscfg (etcnagios by default) and add the client config file

For eg

rootcentos7-vm confd tail etcnagiosnagioscfg

cfg_file=etcnagiosobjectsnrpe-test1cfg

3 Now verify the config and fix any errors

nagios -v etcnagiosnagioscfg

4 Restart services

systemctl restart nagios

systemctl restart nrpe

systemctl restart httpd

Solace Nagios Toolset Integration

24

74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot

below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status

75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and

differences

1 NCSA does passive check This mean client choses when to send the status to the server This allowes

fine grain control over the update timing and useful for near real time updates

2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically

This leads to simpler install

3 NRPE does active check The server pulls the info from the client periodically This is better suited for

centrally adminsterored policies

4 NRPE requires a daemon be runnion on all the clients to service NRPE requests

5 NCSA is not in active development anymore (no updates since 2013)

Solace Nagios Toolset Integration

25

8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD

(Round Robin Database) It can collect manage and display graphs of collected data

81 Prerequisites

811 Cacti Dependencies

Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use

MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration

unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not

required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files

82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are

instructions and materials available that can be readily used Here are some lnks

httpdocscactinetmanual0881_installation1_install_unix

httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-

fedora-17-12

httpwwwcactinetdownloadsdocshtmldebugginghtml

83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to

Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for

illustration which can be replaced with any monitoring using right scripts

Solace Nagios Toolset Integration

26

831 Add Custom Data Input method

Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the

solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script

need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected

form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot

do any validation on Output Field

8311 Data Input

8312 Input

8313 Output

Solace Nagios Toolset Integration

27

832 Add Custom Data Template

Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to

declare info for the specific queue (router name vpn name queue name and credentials)

Solace Nagios Toolset Integration

28

833 Add a custom Graph Template

Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph

Template Items)

Solace Nagios Toolset Integration

29

834 Add Solace Appliance as Device

Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to

run host specific services like ping test

8341 Add Data Source List

Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown

here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the

correspoinding RRD files for these resources

Solace Nagios Toolset Integration

30

8342 Add Graph List

Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the

specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool

Solace Nagios Toolset Integration

31

84 Verification

841 Basic Install

Once you have basic Cacti setup up and running you should be able to access the install and do further configuration

thru web admin page at

httpltyour-server-ipgtcacti

After configuration you would be able to see basic system metrics for the localhost that would look similar to this

Solace Nagios Toolset Integration

32

842 Solace host check

Once Solce router is added as a device from he Devices link you would be able view the router with basic health test

such as ping status

843 Solace VPN Metrics Sample

Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue

statistics showing queue depth for a single queue on a VPN

Solace Nagios Toolset Integration

33

As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of

aggregation of queue stats over various window period from hour to year

Solace Nagios Toolset Integration

34

9 Additional Topics

91 Admin Tasks

911 Creating custom Maps

Create custom maps for any outputs that need it for example turn the iface counter values into gauge

Add to usrlocalnagiosgraphetcnagiosgraphmap

Service type iface

outputIFACE OK - [Interface161EnabledyesLinkyes]

perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-

bytes=363571175913

perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)

and push s [ if_stats

[tx-pkts COUNTER int $1 ]

[rx-pkts COUNTER int $2 ]

[tx-bytes COUNTER int $3 ]

[rx-bytes COUNTER int $4 ] ]

912 Adding additional routers VPNs

Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run

mk_nagiossolacecfg to generate required solace plugin Config files

Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be

added into etcnagiosnagioscfg file

Note that the services need to be restarted for the changes to be effective

913 Putting Nagios on Verbose mode

Change debug level in Nagios config to 2048

grep ^debug_level etcnagiosnagioscfg

debug_level=2048

Now check the debug logs All commands executed and output will be logged over there

Solace Nagios Toolset Integration

35

tail varlognagiosnagiosdebug

[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING

[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w

usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --

queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet

[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w

usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --

vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet

[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING

914 Modifying iptables

When you Cant access http from outside or when telnet to server on port 80 fail

Check Iptables routes

iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT

iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT

iptables-save | tee etcsysconfigiptables

service iptables save

service iptables restart

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

2

Table of Contents

Contents Table of Contents 2 1 Overview 4

11 Audience 5 12 Related Documentation 5 13 Prerequisites 5

131 Requirements 5 132 Assumptions 5

2 Why Solace 6 Superior Performance 6 Robustness 6 Simple Architecture 6 Simple Operations 6 Cost Savings 6

3 Nagios Setup 7 31 Prerequisites 7 32 Installation 7 33 Post install setup 8

331 HTTP Setup 8 332 Firewall setup 8 333 Nagios Admin User 8

34 Verification 9 4 Nagiosgraph Setup 10

41 Prerequisites 10 42 Installation 10 43 Post install setup 11

431 Restarting Services 11 44 Verification 12

5 Solace Nagios Plugin Setup 14 51 Prerequisites 14 52 Installation 14 53 Post Install Setup 16 54 Verification 16

6 NSCA Setup 18 61 Prerequisites 18 62 Installation 18 63 Post install setup 19 64 Verification 20

641 Sending Host Alert 20 642 Sending Service Alert 21

7 NRPE Setup 22 71 Prerequisites 22 72 Installation 22

721 Server (CentOS) 22 722 Client (Ubuntu) 22

73 Post Install Configuration 23 731 Client Configuration 23 732 Server Configuration 23

74 Verification 24 75 NCSA vs NRPE 24

8 Cacti Setup 25 81 Prerequisites 25

811 Cacti Dependencies 25 82 Installation 25 83 Post Install Setup 25

831 Add Custom Data Input method 26 832 Add Custom Data Template 27 833 Add a custom Graph Template 28

Solace Nagios Toolset Integration

3

834 Add Solace Appliance as Device 29 84 Verification 31

841 Basic Install 31 842 Solace host check 32 843 Solace VPN Metrics Sample 32

9 Additional Topics 34 91 Admin Tasks 34

911 Creating custom Maps 34 912 Adding additional routers VPNs 34 913 Putting Nagios on Verbose mode 34 914 Modifying iptables 35

92 Troubleshooting 36 921 Nagios not starting up 36 922 Canrsquot connect to HTTP (or other port) from outside 36 923 CGI Error in showcgi 36 924 CGI Error in Solace graphs 37 925 Canrsquot stat command file error 37

10 Setup Scripts and Config files 38 101 Install Info 38 102 Nagios 39

1021 setup_nagiossolace 39 1022 mk_nagiossolacecfg 45 1023 Solace Nagios Config (cfgsolacecfg) 47 1024 Template router (cfgsolace-templateroutercfg) 50 1025 Template and Sample router Config 53

103 Solace Nagios Plugin 53 104 NSCA 54

1041 setup_nsca 54 1042 send_ngalertpl 57 1043 nsca_init 61 1044 NSCA Config 63

105 NRPE 64 1051 Sample server config file 64 1052 Sample Client Config 65

Solace Nagios Toolset Integration

4

1 Overview Nagios (previously NetSaint) is an open-source monitoring and alerting system thatrsquos widely used to monitor systems

networks and infrastructure Objects monitored by Nagios are split into two categories hosts (physical machines) and

services (particular functionalities) Nagios does not perform any host or service checks on its own and relies on plugins

to do this This makes it a very modular and flexible solution

Extending this ldquomonitoring by pluginrdquo paradigm of Nagios Solace provides a set of plugin scripts that perform the

Appliance and VPN monitoring Following Nagios convention the plugins add host (router) and service (VPN

resources) categories

Nagiograph parses output and performance data from Nagios plugins and generates graphs and HTML pages for

reporting Nagiograph stores data in RRD (Round Robin Database) files as time series data RRDTool is an Open

source parser that integrates into Shell scripts Perl Python Ruby Tcl etc

The following picture illustrates different components and high level data flow

Solace Nagios Toolset Integration

5

11 Audience This document is primarily intended for Solace administrators and operations teams responsible for setting up and

monitoring infrastructure

12 Related Documentation These documents contain information related to the feature defined in this document

Document ID Document Title Document Source

[Solace-Portal] Solace Developer Portal httpdevsolacesystemscom

[Solace-FG] Solace Messaging Platform ndash Feature

Guide

httpdevsolacesystemscomdocsmessaging-

platform-feature-guide

[Solace-FP] Solace Messaging Platform ndash Feature

Provisioning

httpdevsolacesystemscomdocsmessaging-

platform-feature-provisioning

[Solace-CLI] Solace Message Router Command Line

Interface Reference

httpdevsolacesystemscomdocscli-reference

Table 1 - Related Documents

13 Prerequisites

131 Requirements

The installation and setup steps require that you have root access on the CentOS server

The installation and setup steps also require that you have CLI access to the Solace appliance

The Solace appliance should be reachable over network from the CentOS server running Nagios

132 Assumptions

This document is based on SolOS version 711 though earier versions (such as SolOS 62) should work

without any changes

Setup steps in this guide were tested on CentOS 63 CentOS 70 (both 64 Bit Minimal install) Command

syntax and file locations would vary on other flavors of Linux

Solace Nagios Toolset Integration

6

2 Why Solace Solace technology efficiently moves information between all kinds of applications users and devices anywhere in the

world over all kinds of networks Solace makes its state-of-the-art data movement capabilities available via hardware

and software ldquomessage routersrdquo that can meet the needs of any application or deployment environment Solacersquos

unique solution offers unmatched capacity performance robustness and TCO so our customers can focus on seizing

business opportunities instead of building and maintaining complex data distribution infrastructure

Superior Performance Solacersquos hardware and software messaging middleware products can cost-effectively meet the performance needs of

any application with feature parity and interoperability that lets companies start small and scale to support higher

volume or more demanding requirements over time and purpose-built appliances that offer 50-100x higher

performance than any other technology for customers or applications that require extremely high capacity or low

latency

Robustness Solace offers high availability (HA) and disaster recovery (DR) without the need for 3rd party products and fast failover

times no other solution can match Distributing data via dedicated TCP connections ensures an orderly well-behaved

system under load and patented techniques ensure that the performance of publishers and high-speed consumers is

never impacted by slow consumers

Simple Architecture Modern enterprises run applications that demand many kinds of data movement such as persistent messaging web

streaming WAN distribution and cloud-based communications By supporting all kinds of data movement with a unified

platform that can be deployed as a small-footprint software broker or high-capacity rack-mounted appliance Solace lets

architects design an end-to-end infrastructure thatrsquos easy to build applications for integrate with existing technologies

secure and scale

Simple Operations Solacersquos solution features a shared administration framework for all kinds of data movement deployment models and

network environments so itrsquos easy for IT staff to deploy monitor manage and upgrade their Solace-based messaging

environment

Cost Savings Solace reduces expenses with high-capacity hardware flexible software and the ability to deploy the right solution for

each problem Solacersquos support for many kinds of messaging lets you replace multiple messaging products with just

one built-in HA DR WAN and Web functionality eliminate the need for third-party products

Solace Nagios Toolset Integration

7

3 Nagios Setup

31 Prerequisites A working Nagios installatin requires the following packagestools

Httpd (Apache)

RRDTool

GD Grahics library and related Perl modules

NRPE (Nagios Remote Plugin Executor)

yum install -y httpd php gcc glibc glibc-common gd gd-devel make net-snmp wget

yum install rrdtool perl-rrdtool perl-GD

yum install perl-CGI perl-Time-HiRes

1 Enabling EPEL

The steps for enabling Extra Packge Repositor (EPEL) are release specific and please check for your install

CentOS 7

wget httpdlfedoraprojectorgpubepel7x86_64eepel-release-7-5noarchrpm

rpm -ivh epel-release-7-5noarchrpm

CentOS 63

rpm -Uvh httpdlfedoraprojectorgpubepel6x86_64epel-release-6-8noarchrpm

rpm -Uvh httprpmsfamillecolletcomenterpriseremi-release-6rpm

32 Installation

yum -y install nagios nagios-plugins-all nagios-plugins-nrpe nrpe

Solace Nagios Toolset Integration

8

33 Post install setup

331 HTTP Setup

Setting up an HTTP server (such as Apache) is beyond the scope of this document Pl refer to the documentation for

your HTTP server Nagios setup should have dropped the required config file to setup URL Aliases The file is typically

ltOptioalConfigDirgtnagioscfg (Eg etchttpdconfdnagioscfg)

If Checking Webadmin below doesnrsquot work check the httpd config and logs to see if this file was processed

332 Firewall setup

Firewall setup and making changes to existing firewall configuration is again beyond the scope of this document For

eg if you are using iptables (default firewall on CentOS) then you may need to explicitly allow incoming connection on

required ports (Refer Additional Topics Section)

333 Nagios Admin User

Create password for nagiosadmin user for web access

htpasswd ndashc etcnagiospasswd nagiosadmin

systemctl restart httpd

systemctl restart nagios

Solace Nagios Toolset Integration

9

34 Verification Point the webbrowser to httpyour-server-ip[port]nagios

Login with the username nagiosadmin and the password above step (htpassword command)

Sample page is shown below

Solace Nagios Toolset Integration

10

4 Nagiosgraph Setup

41 Prerequisites NagiosGraph installation requires the following packagestools

RRDTool and related Perl modules

GD Graphics Library and related Perl modules

yum install rrdtool perl-rrdtool perl-GD

42 Installation

cd optpkg or path of your choice

wget httpdownloadssourceforgenetprojectnagiosgraphnagiosgraph152nagiosgraph-

152targzwget

tar xvzf nagiosgraph-152targz

cd nagiosgraph-152

installpl --check-prereq

hellip

installpl --layout standalone --prefix usrlocalnagiosgraph --nagios-perfdata-

file=varlognagiosperfdatalog --var-dir=varnagios

hellip

hellip

Continue with this configuration [y] y

hellip

Solace Nagios Toolset Integration

11

43 Post install setup Run the Solace provided Nagios that modify the Config files appropriately Edit the script and change the file paths if

necessary at the top of the script

Running setup_nagiossolace can be deferred until Solace plugins are installed (in following section)If Solace specific

tools are not in place or the Config files not setup corresponding steps will be skipped

431 Restarting Services

chkconfig httpd on ampamp chkconfig nagios on

systemctl restart httpd

systemctl restart nagios

setup_nagiossolace

Checking required files

hellip

Setting up config files

hellip

Restarting services

nagios

nagios (pid 4155) is running

httpd

hellip

Solace Nagios Toolset Integration

12

44 Verification The above script adds another host by name ldquolocalhost-grrdquo with graphics enabled that can be used to test the

Nagiosgraph install Sample page is shown below

Solace Nagios Toolset Integration

13

Clicking on the graph symbols to the right of service name brings up a graph with various time bracket A sample graph

for one of the services (HTTP) for a day is shown below

Solace Nagios Toolset Integration

14

5 Solace Nagios Plugin Setup

51 Prerequisites Solace Nagios plugin requirs the following tools packages

Perl modules to handle XML amp Nagios

Solace pluging package

yum install perl-CPAN perl-XML-LibXML perl-Nagios-Plugin

perl -MCPAN -e shell

cpangt install NagiosPlugin

cpangt install NagiosPluginDieNicely

cpangt install LWPUserAgent

cpangt install NagiosConfig

cpangt exit

52 Installation

1 Create Sample file

Create a sample router Config This file used by the setup script (below) to create commands for this config This step

can also be repeated for other routersvpns using the command mk_nagiossolacecfg script

cat cfgsolace-vmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=19216856102

HOSTPORT=8080

USERNAME=admin

PASSWORD=

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

Solace Nagios Toolset Integration

15

2 Setup Plugin

Run the setup_nagiossolace script This will setup both Nagiosgraph and the solace plugin If Nagisgraph is already

setup the related steps will be skipped

mk_nagiossolacecfg cfgsolace-vmr2cfg

Generating Nagios config for Solace appliance

Using cfgfile cfgsolace-vmr2cfg

Using template file cfgsolace-templaterouter-no_bi_bridgecfg

Generating config

etcnagiosobjectssolace-solace-vmr2cfg

Setting up config files

etcnagiosnagioscfg

Solace Nagios Toolset Integration

16

53 Post Install Setup

54 Verification If the sample router info is successfully configured a host entry matching the appliance hostname from Config file will

be added to the web page

A sample entry is shown below

Solace Nagios Toolset Integration

17

Clicking on the graph icon to the right of the service name brings up the graphs with options A sample graph is shown

below

Solace Nagios Toolset Integration

18

6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the

external command file Nagios would periodically check the external command file and process them This ldquopassive

checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured

hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status

updates and alerting

61 Prerequisites

62 Installation

1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291

(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)

2 Untar and follow install instructions in the package

$ cd optpkg (or directory of your choice)

$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site

for latest package name and replace)

$ tar xvzf nsca-291targz (adjust version accordingly)

$ cd optpkgnsca-291 (adjust version accordingly)

$ configure

Configuration summary for nsca 291 01-27-2012

General Options

-------------------------

NSCA port 5667

NSCA user nagios

NSCA group nagios

$ make all

Solace Nagios Toolset Integration

19

63 Post install setup

1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca

This will generate required Config file and setup nsca as service

NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP

setup_nsca -s optpkgnsca-291

srcdir optpkgnsca-291 ok

target dir usrlocalnsca ok

Checking

hellip

Installing

hellip

Creating service

Installation complete

2 Start service

Now that the component is installed start it up

systemctl restart nsca

Solace Nagios Toolset Integration

20

64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl

ndashh for usage info

641 Sending Host Alert

send_ngalertpl -n vmr3 -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending host command vmr30 - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0

Output - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe

[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection

Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -

[Status=UP]

Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]

Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]

Solace Nagios Toolset Integration

21

642 Sending Service Alert

send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending service command vmr3hardware0 - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service

Description hardware Return Code 0 Output - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe

Nov 9 154116 centos7-vm nsca[9888] End of connection

Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND

PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]

Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]

Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]

Solace Nagios Toolset Integration

22

7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing

monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an

active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be

configured on both the server and the client end though the client end changes are relatively light

In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server

as client

71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that

network connectivity exists between the server and client hosts and the ports are open both ways

72 Installation

721 Server (CentOS)

In addition to Nagios server packages we would need to install the following These were already covered in previous

sections and listed here for completeness and for any standalone installs

NRPE

Nagios Pluggins

OpenSSL

yum install -y nrpe nagios-plugins-all openssl

722 Client (Ubuntu)

On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component

apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-

standard

Solace Nagios Toolset Integration

23

73 Post Install Configuration

731 Client Configuration

1 Add Nagios server IP to allowed_hosts in nrpecfg

grep ^allowed_hosts etcnagiosnrpecfg

allowed_hosts=127001192168562011921681223

2 Restart service

systemctl start nagios-nrpe-server

732 Server Configuration

1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in

ldquoSetup Scripts and Config Filesrdquo section

2 Update nagioscfg (etcnagios by default) and add the client config file

For eg

rootcentos7-vm confd tail etcnagiosnagioscfg

cfg_file=etcnagiosobjectsnrpe-test1cfg

3 Now verify the config and fix any errors

nagios -v etcnagiosnagioscfg

4 Restart services

systemctl restart nagios

systemctl restart nrpe

systemctl restart httpd

Solace Nagios Toolset Integration

24

74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot

below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status

75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and

differences

1 NCSA does passive check This mean client choses when to send the status to the server This allowes

fine grain control over the update timing and useful for near real time updates

2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically

This leads to simpler install

3 NRPE does active check The server pulls the info from the client periodically This is better suited for

centrally adminsterored policies

4 NRPE requires a daemon be runnion on all the clients to service NRPE requests

5 NCSA is not in active development anymore (no updates since 2013)

Solace Nagios Toolset Integration

25

8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD

(Round Robin Database) It can collect manage and display graphs of collected data

81 Prerequisites

811 Cacti Dependencies

Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use

MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration

unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not

required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files

82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are

instructions and materials available that can be readily used Here are some lnks

httpdocscactinetmanual0881_installation1_install_unix

httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-

fedora-17-12

httpwwwcactinetdownloadsdocshtmldebugginghtml

83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to

Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for

illustration which can be replaced with any monitoring using right scripts

Solace Nagios Toolset Integration

26

831 Add Custom Data Input method

Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the

solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script

need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected

form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot

do any validation on Output Field

8311 Data Input

8312 Input

8313 Output

Solace Nagios Toolset Integration

27

832 Add Custom Data Template

Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to

declare info for the specific queue (router name vpn name queue name and credentials)

Solace Nagios Toolset Integration

28

833 Add a custom Graph Template

Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph

Template Items)

Solace Nagios Toolset Integration

29

834 Add Solace Appliance as Device

Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to

run host specific services like ping test

8341 Add Data Source List

Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown

here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the

correspoinding RRD files for these resources

Solace Nagios Toolset Integration

30

8342 Add Graph List

Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the

specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool

Solace Nagios Toolset Integration

31

84 Verification

841 Basic Install

Once you have basic Cacti setup up and running you should be able to access the install and do further configuration

thru web admin page at

httpltyour-server-ipgtcacti

After configuration you would be able to see basic system metrics for the localhost that would look similar to this

Solace Nagios Toolset Integration

32

842 Solace host check

Once Solce router is added as a device from he Devices link you would be able view the router with basic health test

such as ping status

843 Solace VPN Metrics Sample

Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue

statistics showing queue depth for a single queue on a VPN

Solace Nagios Toolset Integration

33

As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of

aggregation of queue stats over various window period from hour to year

Solace Nagios Toolset Integration

34

9 Additional Topics

91 Admin Tasks

911 Creating custom Maps

Create custom maps for any outputs that need it for example turn the iface counter values into gauge

Add to usrlocalnagiosgraphetcnagiosgraphmap

Service type iface

outputIFACE OK - [Interface161EnabledyesLinkyes]

perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-

bytes=363571175913

perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)

and push s [ if_stats

[tx-pkts COUNTER int $1 ]

[rx-pkts COUNTER int $2 ]

[tx-bytes COUNTER int $3 ]

[rx-bytes COUNTER int $4 ] ]

912 Adding additional routers VPNs

Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run

mk_nagiossolacecfg to generate required solace plugin Config files

Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be

added into etcnagiosnagioscfg file

Note that the services need to be restarted for the changes to be effective

913 Putting Nagios on Verbose mode

Change debug level in Nagios config to 2048

grep ^debug_level etcnagiosnagioscfg

debug_level=2048

Now check the debug logs All commands executed and output will be logged over there

Solace Nagios Toolset Integration

35

tail varlognagiosnagiosdebug

[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING

[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w

usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --

queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet

[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w

usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --

vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet

[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING

914 Modifying iptables

When you Cant access http from outside or when telnet to server on port 80 fail

Check Iptables routes

iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT

iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT

iptables-save | tee etcsysconfigiptables

service iptables save

service iptables restart

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

3

834 Add Solace Appliance as Device 29 84 Verification 31

841 Basic Install 31 842 Solace host check 32 843 Solace VPN Metrics Sample 32

9 Additional Topics 34 91 Admin Tasks 34

911 Creating custom Maps 34 912 Adding additional routers VPNs 34 913 Putting Nagios on Verbose mode 34 914 Modifying iptables 35

92 Troubleshooting 36 921 Nagios not starting up 36 922 Canrsquot connect to HTTP (or other port) from outside 36 923 CGI Error in showcgi 36 924 CGI Error in Solace graphs 37 925 Canrsquot stat command file error 37

10 Setup Scripts and Config files 38 101 Install Info 38 102 Nagios 39

1021 setup_nagiossolace 39 1022 mk_nagiossolacecfg 45 1023 Solace Nagios Config (cfgsolacecfg) 47 1024 Template router (cfgsolace-templateroutercfg) 50 1025 Template and Sample router Config 53

103 Solace Nagios Plugin 53 104 NSCA 54

1041 setup_nsca 54 1042 send_ngalertpl 57 1043 nsca_init 61 1044 NSCA Config 63

105 NRPE 64 1051 Sample server config file 64 1052 Sample Client Config 65

Solace Nagios Toolset Integration

4

1 Overview Nagios (previously NetSaint) is an open-source monitoring and alerting system thatrsquos widely used to monitor systems

networks and infrastructure Objects monitored by Nagios are split into two categories hosts (physical machines) and

services (particular functionalities) Nagios does not perform any host or service checks on its own and relies on plugins

to do this This makes it a very modular and flexible solution

Extending this ldquomonitoring by pluginrdquo paradigm of Nagios Solace provides a set of plugin scripts that perform the

Appliance and VPN monitoring Following Nagios convention the plugins add host (router) and service (VPN

resources) categories

Nagiograph parses output and performance data from Nagios plugins and generates graphs and HTML pages for

reporting Nagiograph stores data in RRD (Round Robin Database) files as time series data RRDTool is an Open

source parser that integrates into Shell scripts Perl Python Ruby Tcl etc

The following picture illustrates different components and high level data flow

Solace Nagios Toolset Integration

5

11 Audience This document is primarily intended for Solace administrators and operations teams responsible for setting up and

monitoring infrastructure

12 Related Documentation These documents contain information related to the feature defined in this document

Document ID Document Title Document Source

[Solace-Portal] Solace Developer Portal httpdevsolacesystemscom

[Solace-FG] Solace Messaging Platform ndash Feature

Guide

httpdevsolacesystemscomdocsmessaging-

platform-feature-guide

[Solace-FP] Solace Messaging Platform ndash Feature

Provisioning

httpdevsolacesystemscomdocsmessaging-

platform-feature-provisioning

[Solace-CLI] Solace Message Router Command Line

Interface Reference

httpdevsolacesystemscomdocscli-reference

Table 1 - Related Documents

13 Prerequisites

131 Requirements

The installation and setup steps require that you have root access on the CentOS server

The installation and setup steps also require that you have CLI access to the Solace appliance

The Solace appliance should be reachable over network from the CentOS server running Nagios

132 Assumptions

This document is based on SolOS version 711 though earier versions (such as SolOS 62) should work

without any changes

Setup steps in this guide were tested on CentOS 63 CentOS 70 (both 64 Bit Minimal install) Command

syntax and file locations would vary on other flavors of Linux

Solace Nagios Toolset Integration

6

2 Why Solace Solace technology efficiently moves information between all kinds of applications users and devices anywhere in the

world over all kinds of networks Solace makes its state-of-the-art data movement capabilities available via hardware

and software ldquomessage routersrdquo that can meet the needs of any application or deployment environment Solacersquos

unique solution offers unmatched capacity performance robustness and TCO so our customers can focus on seizing

business opportunities instead of building and maintaining complex data distribution infrastructure

Superior Performance Solacersquos hardware and software messaging middleware products can cost-effectively meet the performance needs of

any application with feature parity and interoperability that lets companies start small and scale to support higher

volume or more demanding requirements over time and purpose-built appliances that offer 50-100x higher

performance than any other technology for customers or applications that require extremely high capacity or low

latency

Robustness Solace offers high availability (HA) and disaster recovery (DR) without the need for 3rd party products and fast failover

times no other solution can match Distributing data via dedicated TCP connections ensures an orderly well-behaved

system under load and patented techniques ensure that the performance of publishers and high-speed consumers is

never impacted by slow consumers

Simple Architecture Modern enterprises run applications that demand many kinds of data movement such as persistent messaging web

streaming WAN distribution and cloud-based communications By supporting all kinds of data movement with a unified

platform that can be deployed as a small-footprint software broker or high-capacity rack-mounted appliance Solace lets

architects design an end-to-end infrastructure thatrsquos easy to build applications for integrate with existing technologies

secure and scale

Simple Operations Solacersquos solution features a shared administration framework for all kinds of data movement deployment models and

network environments so itrsquos easy for IT staff to deploy monitor manage and upgrade their Solace-based messaging

environment

Cost Savings Solace reduces expenses with high-capacity hardware flexible software and the ability to deploy the right solution for

each problem Solacersquos support for many kinds of messaging lets you replace multiple messaging products with just

one built-in HA DR WAN and Web functionality eliminate the need for third-party products

Solace Nagios Toolset Integration

7

3 Nagios Setup

31 Prerequisites A working Nagios installatin requires the following packagestools

Httpd (Apache)

RRDTool

GD Grahics library and related Perl modules

NRPE (Nagios Remote Plugin Executor)

yum install -y httpd php gcc glibc glibc-common gd gd-devel make net-snmp wget

yum install rrdtool perl-rrdtool perl-GD

yum install perl-CGI perl-Time-HiRes

1 Enabling EPEL

The steps for enabling Extra Packge Repositor (EPEL) are release specific and please check for your install

CentOS 7

wget httpdlfedoraprojectorgpubepel7x86_64eepel-release-7-5noarchrpm

rpm -ivh epel-release-7-5noarchrpm

CentOS 63

rpm -Uvh httpdlfedoraprojectorgpubepel6x86_64epel-release-6-8noarchrpm

rpm -Uvh httprpmsfamillecolletcomenterpriseremi-release-6rpm

32 Installation

yum -y install nagios nagios-plugins-all nagios-plugins-nrpe nrpe

Solace Nagios Toolset Integration

8

33 Post install setup

331 HTTP Setup

Setting up an HTTP server (such as Apache) is beyond the scope of this document Pl refer to the documentation for

your HTTP server Nagios setup should have dropped the required config file to setup URL Aliases The file is typically

ltOptioalConfigDirgtnagioscfg (Eg etchttpdconfdnagioscfg)

If Checking Webadmin below doesnrsquot work check the httpd config and logs to see if this file was processed

332 Firewall setup

Firewall setup and making changes to existing firewall configuration is again beyond the scope of this document For

eg if you are using iptables (default firewall on CentOS) then you may need to explicitly allow incoming connection on

required ports (Refer Additional Topics Section)

333 Nagios Admin User

Create password for nagiosadmin user for web access

htpasswd ndashc etcnagiospasswd nagiosadmin

systemctl restart httpd

systemctl restart nagios

Solace Nagios Toolset Integration

9

34 Verification Point the webbrowser to httpyour-server-ip[port]nagios

Login with the username nagiosadmin and the password above step (htpassword command)

Sample page is shown below

Solace Nagios Toolset Integration

10

4 Nagiosgraph Setup

41 Prerequisites NagiosGraph installation requires the following packagestools

RRDTool and related Perl modules

GD Graphics Library and related Perl modules

yum install rrdtool perl-rrdtool perl-GD

42 Installation

cd optpkg or path of your choice

wget httpdownloadssourceforgenetprojectnagiosgraphnagiosgraph152nagiosgraph-

152targzwget

tar xvzf nagiosgraph-152targz

cd nagiosgraph-152

installpl --check-prereq

hellip

installpl --layout standalone --prefix usrlocalnagiosgraph --nagios-perfdata-

file=varlognagiosperfdatalog --var-dir=varnagios

hellip

hellip

Continue with this configuration [y] y

hellip

Solace Nagios Toolset Integration

11

43 Post install setup Run the Solace provided Nagios that modify the Config files appropriately Edit the script and change the file paths if

necessary at the top of the script

Running setup_nagiossolace can be deferred until Solace plugins are installed (in following section)If Solace specific

tools are not in place or the Config files not setup corresponding steps will be skipped

431 Restarting Services

chkconfig httpd on ampamp chkconfig nagios on

systemctl restart httpd

systemctl restart nagios

setup_nagiossolace

Checking required files

hellip

Setting up config files

hellip

Restarting services

nagios

nagios (pid 4155) is running

httpd

hellip

Solace Nagios Toolset Integration

12

44 Verification The above script adds another host by name ldquolocalhost-grrdquo with graphics enabled that can be used to test the

Nagiosgraph install Sample page is shown below

Solace Nagios Toolset Integration

13

Clicking on the graph symbols to the right of service name brings up a graph with various time bracket A sample graph

for one of the services (HTTP) for a day is shown below

Solace Nagios Toolset Integration

14

5 Solace Nagios Plugin Setup

51 Prerequisites Solace Nagios plugin requirs the following tools packages

Perl modules to handle XML amp Nagios

Solace pluging package

yum install perl-CPAN perl-XML-LibXML perl-Nagios-Plugin

perl -MCPAN -e shell

cpangt install NagiosPlugin

cpangt install NagiosPluginDieNicely

cpangt install LWPUserAgent

cpangt install NagiosConfig

cpangt exit

52 Installation

1 Create Sample file

Create a sample router Config This file used by the setup script (below) to create commands for this config This step

can also be repeated for other routersvpns using the command mk_nagiossolacecfg script

cat cfgsolace-vmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=19216856102

HOSTPORT=8080

USERNAME=admin

PASSWORD=

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

Solace Nagios Toolset Integration

15

2 Setup Plugin

Run the setup_nagiossolace script This will setup both Nagiosgraph and the solace plugin If Nagisgraph is already

setup the related steps will be skipped

mk_nagiossolacecfg cfgsolace-vmr2cfg

Generating Nagios config for Solace appliance

Using cfgfile cfgsolace-vmr2cfg

Using template file cfgsolace-templaterouter-no_bi_bridgecfg

Generating config

etcnagiosobjectssolace-solace-vmr2cfg

Setting up config files

etcnagiosnagioscfg

Solace Nagios Toolset Integration

16

53 Post Install Setup

54 Verification If the sample router info is successfully configured a host entry matching the appliance hostname from Config file will

be added to the web page

A sample entry is shown below

Solace Nagios Toolset Integration

17

Clicking on the graph icon to the right of the service name brings up the graphs with options A sample graph is shown

below

Solace Nagios Toolset Integration

18

6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the

external command file Nagios would periodically check the external command file and process them This ldquopassive

checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured

hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status

updates and alerting

61 Prerequisites

62 Installation

1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291

(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)

2 Untar and follow install instructions in the package

$ cd optpkg (or directory of your choice)

$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site

for latest package name and replace)

$ tar xvzf nsca-291targz (adjust version accordingly)

$ cd optpkgnsca-291 (adjust version accordingly)

$ configure

Configuration summary for nsca 291 01-27-2012

General Options

-------------------------

NSCA port 5667

NSCA user nagios

NSCA group nagios

$ make all

Solace Nagios Toolset Integration

19

63 Post install setup

1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca

This will generate required Config file and setup nsca as service

NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP

setup_nsca -s optpkgnsca-291

srcdir optpkgnsca-291 ok

target dir usrlocalnsca ok

Checking

hellip

Installing

hellip

Creating service

Installation complete

2 Start service

Now that the component is installed start it up

systemctl restart nsca

Solace Nagios Toolset Integration

20

64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl

ndashh for usage info

641 Sending Host Alert

send_ngalertpl -n vmr3 -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending host command vmr30 - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0

Output - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe

[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection

Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -

[Status=UP]

Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]

Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]

Solace Nagios Toolset Integration

21

642 Sending Service Alert

send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending service command vmr3hardware0 - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service

Description hardware Return Code 0 Output - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe

Nov 9 154116 centos7-vm nsca[9888] End of connection

Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND

PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]

Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]

Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]

Solace Nagios Toolset Integration

22

7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing

monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an

active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be

configured on both the server and the client end though the client end changes are relatively light

In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server

as client

71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that

network connectivity exists between the server and client hosts and the ports are open both ways

72 Installation

721 Server (CentOS)

In addition to Nagios server packages we would need to install the following These were already covered in previous

sections and listed here for completeness and for any standalone installs

NRPE

Nagios Pluggins

OpenSSL

yum install -y nrpe nagios-plugins-all openssl

722 Client (Ubuntu)

On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component

apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-

standard

Solace Nagios Toolset Integration

23

73 Post Install Configuration

731 Client Configuration

1 Add Nagios server IP to allowed_hosts in nrpecfg

grep ^allowed_hosts etcnagiosnrpecfg

allowed_hosts=127001192168562011921681223

2 Restart service

systemctl start nagios-nrpe-server

732 Server Configuration

1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in

ldquoSetup Scripts and Config Filesrdquo section

2 Update nagioscfg (etcnagios by default) and add the client config file

For eg

rootcentos7-vm confd tail etcnagiosnagioscfg

cfg_file=etcnagiosobjectsnrpe-test1cfg

3 Now verify the config and fix any errors

nagios -v etcnagiosnagioscfg

4 Restart services

systemctl restart nagios

systemctl restart nrpe

systemctl restart httpd

Solace Nagios Toolset Integration

24

74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot

below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status

75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and

differences

1 NCSA does passive check This mean client choses when to send the status to the server This allowes

fine grain control over the update timing and useful for near real time updates

2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically

This leads to simpler install

3 NRPE does active check The server pulls the info from the client periodically This is better suited for

centrally adminsterored policies

4 NRPE requires a daemon be runnion on all the clients to service NRPE requests

5 NCSA is not in active development anymore (no updates since 2013)

Solace Nagios Toolset Integration

25

8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD

(Round Robin Database) It can collect manage and display graphs of collected data

81 Prerequisites

811 Cacti Dependencies

Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use

MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration

unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not

required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files

82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are

instructions and materials available that can be readily used Here are some lnks

httpdocscactinetmanual0881_installation1_install_unix

httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-

fedora-17-12

httpwwwcactinetdownloadsdocshtmldebugginghtml

83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to

Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for

illustration which can be replaced with any monitoring using right scripts

Solace Nagios Toolset Integration

26

831 Add Custom Data Input method

Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the

solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script

need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected

form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot

do any validation on Output Field

8311 Data Input

8312 Input

8313 Output

Solace Nagios Toolset Integration

27

832 Add Custom Data Template

Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to

declare info for the specific queue (router name vpn name queue name and credentials)

Solace Nagios Toolset Integration

28

833 Add a custom Graph Template

Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph

Template Items)

Solace Nagios Toolset Integration

29

834 Add Solace Appliance as Device

Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to

run host specific services like ping test

8341 Add Data Source List

Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown

here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the

correspoinding RRD files for these resources

Solace Nagios Toolset Integration

30

8342 Add Graph List

Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the

specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool

Solace Nagios Toolset Integration

31

84 Verification

841 Basic Install

Once you have basic Cacti setup up and running you should be able to access the install and do further configuration

thru web admin page at

httpltyour-server-ipgtcacti

After configuration you would be able to see basic system metrics for the localhost that would look similar to this

Solace Nagios Toolset Integration

32

842 Solace host check

Once Solce router is added as a device from he Devices link you would be able view the router with basic health test

such as ping status

843 Solace VPN Metrics Sample

Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue

statistics showing queue depth for a single queue on a VPN

Solace Nagios Toolset Integration

33

As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of

aggregation of queue stats over various window period from hour to year

Solace Nagios Toolset Integration

34

9 Additional Topics

91 Admin Tasks

911 Creating custom Maps

Create custom maps for any outputs that need it for example turn the iface counter values into gauge

Add to usrlocalnagiosgraphetcnagiosgraphmap

Service type iface

outputIFACE OK - [Interface161EnabledyesLinkyes]

perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-

bytes=363571175913

perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)

and push s [ if_stats

[tx-pkts COUNTER int $1 ]

[rx-pkts COUNTER int $2 ]

[tx-bytes COUNTER int $3 ]

[rx-bytes COUNTER int $4 ] ]

912 Adding additional routers VPNs

Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run

mk_nagiossolacecfg to generate required solace plugin Config files

Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be

added into etcnagiosnagioscfg file

Note that the services need to be restarted for the changes to be effective

913 Putting Nagios on Verbose mode

Change debug level in Nagios config to 2048

grep ^debug_level etcnagiosnagioscfg

debug_level=2048

Now check the debug logs All commands executed and output will be logged over there

Solace Nagios Toolset Integration

35

tail varlognagiosnagiosdebug

[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING

[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w

usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --

queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet

[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w

usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --

vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet

[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING

914 Modifying iptables

When you Cant access http from outside or when telnet to server on port 80 fail

Check Iptables routes

iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT

iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT

iptables-save | tee etcsysconfigiptables

service iptables save

service iptables restart

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

4

1 Overview Nagios (previously NetSaint) is an open-source monitoring and alerting system thatrsquos widely used to monitor systems

networks and infrastructure Objects monitored by Nagios are split into two categories hosts (physical machines) and

services (particular functionalities) Nagios does not perform any host or service checks on its own and relies on plugins

to do this This makes it a very modular and flexible solution

Extending this ldquomonitoring by pluginrdquo paradigm of Nagios Solace provides a set of plugin scripts that perform the

Appliance and VPN monitoring Following Nagios convention the plugins add host (router) and service (VPN

resources) categories

Nagiograph parses output and performance data from Nagios plugins and generates graphs and HTML pages for

reporting Nagiograph stores data in RRD (Round Robin Database) files as time series data RRDTool is an Open

source parser that integrates into Shell scripts Perl Python Ruby Tcl etc

The following picture illustrates different components and high level data flow

Solace Nagios Toolset Integration

5

11 Audience This document is primarily intended for Solace administrators and operations teams responsible for setting up and

monitoring infrastructure

12 Related Documentation These documents contain information related to the feature defined in this document

Document ID Document Title Document Source

[Solace-Portal] Solace Developer Portal httpdevsolacesystemscom

[Solace-FG] Solace Messaging Platform ndash Feature

Guide

httpdevsolacesystemscomdocsmessaging-

platform-feature-guide

[Solace-FP] Solace Messaging Platform ndash Feature

Provisioning

httpdevsolacesystemscomdocsmessaging-

platform-feature-provisioning

[Solace-CLI] Solace Message Router Command Line

Interface Reference

httpdevsolacesystemscomdocscli-reference

Table 1 - Related Documents

13 Prerequisites

131 Requirements

The installation and setup steps require that you have root access on the CentOS server

The installation and setup steps also require that you have CLI access to the Solace appliance

The Solace appliance should be reachable over network from the CentOS server running Nagios

132 Assumptions

This document is based on SolOS version 711 though earier versions (such as SolOS 62) should work

without any changes

Setup steps in this guide were tested on CentOS 63 CentOS 70 (both 64 Bit Minimal install) Command

syntax and file locations would vary on other flavors of Linux

Solace Nagios Toolset Integration

6

2 Why Solace Solace technology efficiently moves information between all kinds of applications users and devices anywhere in the

world over all kinds of networks Solace makes its state-of-the-art data movement capabilities available via hardware

and software ldquomessage routersrdquo that can meet the needs of any application or deployment environment Solacersquos

unique solution offers unmatched capacity performance robustness and TCO so our customers can focus on seizing

business opportunities instead of building and maintaining complex data distribution infrastructure

Superior Performance Solacersquos hardware and software messaging middleware products can cost-effectively meet the performance needs of

any application with feature parity and interoperability that lets companies start small and scale to support higher

volume or more demanding requirements over time and purpose-built appliances that offer 50-100x higher

performance than any other technology for customers or applications that require extremely high capacity or low

latency

Robustness Solace offers high availability (HA) and disaster recovery (DR) without the need for 3rd party products and fast failover

times no other solution can match Distributing data via dedicated TCP connections ensures an orderly well-behaved

system under load and patented techniques ensure that the performance of publishers and high-speed consumers is

never impacted by slow consumers

Simple Architecture Modern enterprises run applications that demand many kinds of data movement such as persistent messaging web

streaming WAN distribution and cloud-based communications By supporting all kinds of data movement with a unified

platform that can be deployed as a small-footprint software broker or high-capacity rack-mounted appliance Solace lets

architects design an end-to-end infrastructure thatrsquos easy to build applications for integrate with existing technologies

secure and scale

Simple Operations Solacersquos solution features a shared administration framework for all kinds of data movement deployment models and

network environments so itrsquos easy for IT staff to deploy monitor manage and upgrade their Solace-based messaging

environment

Cost Savings Solace reduces expenses with high-capacity hardware flexible software and the ability to deploy the right solution for

each problem Solacersquos support for many kinds of messaging lets you replace multiple messaging products with just

one built-in HA DR WAN and Web functionality eliminate the need for third-party products

Solace Nagios Toolset Integration

7

3 Nagios Setup

31 Prerequisites A working Nagios installatin requires the following packagestools

Httpd (Apache)

RRDTool

GD Grahics library and related Perl modules

NRPE (Nagios Remote Plugin Executor)

yum install -y httpd php gcc glibc glibc-common gd gd-devel make net-snmp wget

yum install rrdtool perl-rrdtool perl-GD

yum install perl-CGI perl-Time-HiRes

1 Enabling EPEL

The steps for enabling Extra Packge Repositor (EPEL) are release specific and please check for your install

CentOS 7

wget httpdlfedoraprojectorgpubepel7x86_64eepel-release-7-5noarchrpm

rpm -ivh epel-release-7-5noarchrpm

CentOS 63

rpm -Uvh httpdlfedoraprojectorgpubepel6x86_64epel-release-6-8noarchrpm

rpm -Uvh httprpmsfamillecolletcomenterpriseremi-release-6rpm

32 Installation

yum -y install nagios nagios-plugins-all nagios-plugins-nrpe nrpe

Solace Nagios Toolset Integration

8

33 Post install setup

331 HTTP Setup

Setting up an HTTP server (such as Apache) is beyond the scope of this document Pl refer to the documentation for

your HTTP server Nagios setup should have dropped the required config file to setup URL Aliases The file is typically

ltOptioalConfigDirgtnagioscfg (Eg etchttpdconfdnagioscfg)

If Checking Webadmin below doesnrsquot work check the httpd config and logs to see if this file was processed

332 Firewall setup

Firewall setup and making changes to existing firewall configuration is again beyond the scope of this document For

eg if you are using iptables (default firewall on CentOS) then you may need to explicitly allow incoming connection on

required ports (Refer Additional Topics Section)

333 Nagios Admin User

Create password for nagiosadmin user for web access

htpasswd ndashc etcnagiospasswd nagiosadmin

systemctl restart httpd

systemctl restart nagios

Solace Nagios Toolset Integration

9

34 Verification Point the webbrowser to httpyour-server-ip[port]nagios

Login with the username nagiosadmin and the password above step (htpassword command)

Sample page is shown below

Solace Nagios Toolset Integration

10

4 Nagiosgraph Setup

41 Prerequisites NagiosGraph installation requires the following packagestools

RRDTool and related Perl modules

GD Graphics Library and related Perl modules

yum install rrdtool perl-rrdtool perl-GD

42 Installation

cd optpkg or path of your choice

wget httpdownloadssourceforgenetprojectnagiosgraphnagiosgraph152nagiosgraph-

152targzwget

tar xvzf nagiosgraph-152targz

cd nagiosgraph-152

installpl --check-prereq

hellip

installpl --layout standalone --prefix usrlocalnagiosgraph --nagios-perfdata-

file=varlognagiosperfdatalog --var-dir=varnagios

hellip

hellip

Continue with this configuration [y] y

hellip

Solace Nagios Toolset Integration

11

43 Post install setup Run the Solace provided Nagios that modify the Config files appropriately Edit the script and change the file paths if

necessary at the top of the script

Running setup_nagiossolace can be deferred until Solace plugins are installed (in following section)If Solace specific

tools are not in place or the Config files not setup corresponding steps will be skipped

431 Restarting Services

chkconfig httpd on ampamp chkconfig nagios on

systemctl restart httpd

systemctl restart nagios

setup_nagiossolace

Checking required files

hellip

Setting up config files

hellip

Restarting services

nagios

nagios (pid 4155) is running

httpd

hellip

Solace Nagios Toolset Integration

12

44 Verification The above script adds another host by name ldquolocalhost-grrdquo with graphics enabled that can be used to test the

Nagiosgraph install Sample page is shown below

Solace Nagios Toolset Integration

13

Clicking on the graph symbols to the right of service name brings up a graph with various time bracket A sample graph

for one of the services (HTTP) for a day is shown below

Solace Nagios Toolset Integration

14

5 Solace Nagios Plugin Setup

51 Prerequisites Solace Nagios plugin requirs the following tools packages

Perl modules to handle XML amp Nagios

Solace pluging package

yum install perl-CPAN perl-XML-LibXML perl-Nagios-Plugin

perl -MCPAN -e shell

cpangt install NagiosPlugin

cpangt install NagiosPluginDieNicely

cpangt install LWPUserAgent

cpangt install NagiosConfig

cpangt exit

52 Installation

1 Create Sample file

Create a sample router Config This file used by the setup script (below) to create commands for this config This step

can also be repeated for other routersvpns using the command mk_nagiossolacecfg script

cat cfgsolace-vmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=19216856102

HOSTPORT=8080

USERNAME=admin

PASSWORD=

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

Solace Nagios Toolset Integration

15

2 Setup Plugin

Run the setup_nagiossolace script This will setup both Nagiosgraph and the solace plugin If Nagisgraph is already

setup the related steps will be skipped

mk_nagiossolacecfg cfgsolace-vmr2cfg

Generating Nagios config for Solace appliance

Using cfgfile cfgsolace-vmr2cfg

Using template file cfgsolace-templaterouter-no_bi_bridgecfg

Generating config

etcnagiosobjectssolace-solace-vmr2cfg

Setting up config files

etcnagiosnagioscfg

Solace Nagios Toolset Integration

16

53 Post Install Setup

54 Verification If the sample router info is successfully configured a host entry matching the appliance hostname from Config file will

be added to the web page

A sample entry is shown below

Solace Nagios Toolset Integration

17

Clicking on the graph icon to the right of the service name brings up the graphs with options A sample graph is shown

below

Solace Nagios Toolset Integration

18

6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the

external command file Nagios would periodically check the external command file and process them This ldquopassive

checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured

hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status

updates and alerting

61 Prerequisites

62 Installation

1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291

(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)

2 Untar and follow install instructions in the package

$ cd optpkg (or directory of your choice)

$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site

for latest package name and replace)

$ tar xvzf nsca-291targz (adjust version accordingly)

$ cd optpkgnsca-291 (adjust version accordingly)

$ configure

Configuration summary for nsca 291 01-27-2012

General Options

-------------------------

NSCA port 5667

NSCA user nagios

NSCA group nagios

$ make all

Solace Nagios Toolset Integration

19

63 Post install setup

1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca

This will generate required Config file and setup nsca as service

NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP

setup_nsca -s optpkgnsca-291

srcdir optpkgnsca-291 ok

target dir usrlocalnsca ok

Checking

hellip

Installing

hellip

Creating service

Installation complete

2 Start service

Now that the component is installed start it up

systemctl restart nsca

Solace Nagios Toolset Integration

20

64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl

ndashh for usage info

641 Sending Host Alert

send_ngalertpl -n vmr3 -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending host command vmr30 - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0

Output - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe

[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection

Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -

[Status=UP]

Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]

Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]

Solace Nagios Toolset Integration

21

642 Sending Service Alert

send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending service command vmr3hardware0 - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service

Description hardware Return Code 0 Output - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe

Nov 9 154116 centos7-vm nsca[9888] End of connection

Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND

PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]

Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]

Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]

Solace Nagios Toolset Integration

22

7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing

monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an

active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be

configured on both the server and the client end though the client end changes are relatively light

In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server

as client

71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that

network connectivity exists between the server and client hosts and the ports are open both ways

72 Installation

721 Server (CentOS)

In addition to Nagios server packages we would need to install the following These were already covered in previous

sections and listed here for completeness and for any standalone installs

NRPE

Nagios Pluggins

OpenSSL

yum install -y nrpe nagios-plugins-all openssl

722 Client (Ubuntu)

On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component

apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-

standard

Solace Nagios Toolset Integration

23

73 Post Install Configuration

731 Client Configuration

1 Add Nagios server IP to allowed_hosts in nrpecfg

grep ^allowed_hosts etcnagiosnrpecfg

allowed_hosts=127001192168562011921681223

2 Restart service

systemctl start nagios-nrpe-server

732 Server Configuration

1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in

ldquoSetup Scripts and Config Filesrdquo section

2 Update nagioscfg (etcnagios by default) and add the client config file

For eg

rootcentos7-vm confd tail etcnagiosnagioscfg

cfg_file=etcnagiosobjectsnrpe-test1cfg

3 Now verify the config and fix any errors

nagios -v etcnagiosnagioscfg

4 Restart services

systemctl restart nagios

systemctl restart nrpe

systemctl restart httpd

Solace Nagios Toolset Integration

24

74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot

below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status

75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and

differences

1 NCSA does passive check This mean client choses when to send the status to the server This allowes

fine grain control over the update timing and useful for near real time updates

2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically

This leads to simpler install

3 NRPE does active check The server pulls the info from the client periodically This is better suited for

centrally adminsterored policies

4 NRPE requires a daemon be runnion on all the clients to service NRPE requests

5 NCSA is not in active development anymore (no updates since 2013)

Solace Nagios Toolset Integration

25

8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD

(Round Robin Database) It can collect manage and display graphs of collected data

81 Prerequisites

811 Cacti Dependencies

Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use

MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration

unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not

required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files

82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are

instructions and materials available that can be readily used Here are some lnks

httpdocscactinetmanual0881_installation1_install_unix

httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-

fedora-17-12

httpwwwcactinetdownloadsdocshtmldebugginghtml

83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to

Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for

illustration which can be replaced with any monitoring using right scripts

Solace Nagios Toolset Integration

26

831 Add Custom Data Input method

Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the

solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script

need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected

form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot

do any validation on Output Field

8311 Data Input

8312 Input

8313 Output

Solace Nagios Toolset Integration

27

832 Add Custom Data Template

Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to

declare info for the specific queue (router name vpn name queue name and credentials)

Solace Nagios Toolset Integration

28

833 Add a custom Graph Template

Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph

Template Items)

Solace Nagios Toolset Integration

29

834 Add Solace Appliance as Device

Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to

run host specific services like ping test

8341 Add Data Source List

Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown

here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the

correspoinding RRD files for these resources

Solace Nagios Toolset Integration

30

8342 Add Graph List

Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the

specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool

Solace Nagios Toolset Integration

31

84 Verification

841 Basic Install

Once you have basic Cacti setup up and running you should be able to access the install and do further configuration

thru web admin page at

httpltyour-server-ipgtcacti

After configuration you would be able to see basic system metrics for the localhost that would look similar to this

Solace Nagios Toolset Integration

32

842 Solace host check

Once Solce router is added as a device from he Devices link you would be able view the router with basic health test

such as ping status

843 Solace VPN Metrics Sample

Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue

statistics showing queue depth for a single queue on a VPN

Solace Nagios Toolset Integration

33

As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of

aggregation of queue stats over various window period from hour to year

Solace Nagios Toolset Integration

34

9 Additional Topics

91 Admin Tasks

911 Creating custom Maps

Create custom maps for any outputs that need it for example turn the iface counter values into gauge

Add to usrlocalnagiosgraphetcnagiosgraphmap

Service type iface

outputIFACE OK - [Interface161EnabledyesLinkyes]

perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-

bytes=363571175913

perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)

and push s [ if_stats

[tx-pkts COUNTER int $1 ]

[rx-pkts COUNTER int $2 ]

[tx-bytes COUNTER int $3 ]

[rx-bytes COUNTER int $4 ] ]

912 Adding additional routers VPNs

Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run

mk_nagiossolacecfg to generate required solace plugin Config files

Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be

added into etcnagiosnagioscfg file

Note that the services need to be restarted for the changes to be effective

913 Putting Nagios on Verbose mode

Change debug level in Nagios config to 2048

grep ^debug_level etcnagiosnagioscfg

debug_level=2048

Now check the debug logs All commands executed and output will be logged over there

Solace Nagios Toolset Integration

35

tail varlognagiosnagiosdebug

[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING

[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w

usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --

queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet

[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w

usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --

vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet

[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING

914 Modifying iptables

When you Cant access http from outside or when telnet to server on port 80 fail

Check Iptables routes

iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT

iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT

iptables-save | tee etcsysconfigiptables

service iptables save

service iptables restart

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

5

11 Audience This document is primarily intended for Solace administrators and operations teams responsible for setting up and

monitoring infrastructure

12 Related Documentation These documents contain information related to the feature defined in this document

Document ID Document Title Document Source

[Solace-Portal] Solace Developer Portal httpdevsolacesystemscom

[Solace-FG] Solace Messaging Platform ndash Feature

Guide

httpdevsolacesystemscomdocsmessaging-

platform-feature-guide

[Solace-FP] Solace Messaging Platform ndash Feature

Provisioning

httpdevsolacesystemscomdocsmessaging-

platform-feature-provisioning

[Solace-CLI] Solace Message Router Command Line

Interface Reference

httpdevsolacesystemscomdocscli-reference

Table 1 - Related Documents

13 Prerequisites

131 Requirements

The installation and setup steps require that you have root access on the CentOS server

The installation and setup steps also require that you have CLI access to the Solace appliance

The Solace appliance should be reachable over network from the CentOS server running Nagios

132 Assumptions

This document is based on SolOS version 711 though earier versions (such as SolOS 62) should work

without any changes

Setup steps in this guide were tested on CentOS 63 CentOS 70 (both 64 Bit Minimal install) Command

syntax and file locations would vary on other flavors of Linux

Solace Nagios Toolset Integration

6

2 Why Solace Solace technology efficiently moves information between all kinds of applications users and devices anywhere in the

world over all kinds of networks Solace makes its state-of-the-art data movement capabilities available via hardware

and software ldquomessage routersrdquo that can meet the needs of any application or deployment environment Solacersquos

unique solution offers unmatched capacity performance robustness and TCO so our customers can focus on seizing

business opportunities instead of building and maintaining complex data distribution infrastructure

Superior Performance Solacersquos hardware and software messaging middleware products can cost-effectively meet the performance needs of

any application with feature parity and interoperability that lets companies start small and scale to support higher

volume or more demanding requirements over time and purpose-built appliances that offer 50-100x higher

performance than any other technology for customers or applications that require extremely high capacity or low

latency

Robustness Solace offers high availability (HA) and disaster recovery (DR) without the need for 3rd party products and fast failover

times no other solution can match Distributing data via dedicated TCP connections ensures an orderly well-behaved

system under load and patented techniques ensure that the performance of publishers and high-speed consumers is

never impacted by slow consumers

Simple Architecture Modern enterprises run applications that demand many kinds of data movement such as persistent messaging web

streaming WAN distribution and cloud-based communications By supporting all kinds of data movement with a unified

platform that can be deployed as a small-footprint software broker or high-capacity rack-mounted appliance Solace lets

architects design an end-to-end infrastructure thatrsquos easy to build applications for integrate with existing technologies

secure and scale

Simple Operations Solacersquos solution features a shared administration framework for all kinds of data movement deployment models and

network environments so itrsquos easy for IT staff to deploy monitor manage and upgrade their Solace-based messaging

environment

Cost Savings Solace reduces expenses with high-capacity hardware flexible software and the ability to deploy the right solution for

each problem Solacersquos support for many kinds of messaging lets you replace multiple messaging products with just

one built-in HA DR WAN and Web functionality eliminate the need for third-party products

Solace Nagios Toolset Integration

7

3 Nagios Setup

31 Prerequisites A working Nagios installatin requires the following packagestools

Httpd (Apache)

RRDTool

GD Grahics library and related Perl modules

NRPE (Nagios Remote Plugin Executor)

yum install -y httpd php gcc glibc glibc-common gd gd-devel make net-snmp wget

yum install rrdtool perl-rrdtool perl-GD

yum install perl-CGI perl-Time-HiRes

1 Enabling EPEL

The steps for enabling Extra Packge Repositor (EPEL) are release specific and please check for your install

CentOS 7

wget httpdlfedoraprojectorgpubepel7x86_64eepel-release-7-5noarchrpm

rpm -ivh epel-release-7-5noarchrpm

CentOS 63

rpm -Uvh httpdlfedoraprojectorgpubepel6x86_64epel-release-6-8noarchrpm

rpm -Uvh httprpmsfamillecolletcomenterpriseremi-release-6rpm

32 Installation

yum -y install nagios nagios-plugins-all nagios-plugins-nrpe nrpe

Solace Nagios Toolset Integration

8

33 Post install setup

331 HTTP Setup

Setting up an HTTP server (such as Apache) is beyond the scope of this document Pl refer to the documentation for

your HTTP server Nagios setup should have dropped the required config file to setup URL Aliases The file is typically

ltOptioalConfigDirgtnagioscfg (Eg etchttpdconfdnagioscfg)

If Checking Webadmin below doesnrsquot work check the httpd config and logs to see if this file was processed

332 Firewall setup

Firewall setup and making changes to existing firewall configuration is again beyond the scope of this document For

eg if you are using iptables (default firewall on CentOS) then you may need to explicitly allow incoming connection on

required ports (Refer Additional Topics Section)

333 Nagios Admin User

Create password for nagiosadmin user for web access

htpasswd ndashc etcnagiospasswd nagiosadmin

systemctl restart httpd

systemctl restart nagios

Solace Nagios Toolset Integration

9

34 Verification Point the webbrowser to httpyour-server-ip[port]nagios

Login with the username nagiosadmin and the password above step (htpassword command)

Sample page is shown below

Solace Nagios Toolset Integration

10

4 Nagiosgraph Setup

41 Prerequisites NagiosGraph installation requires the following packagestools

RRDTool and related Perl modules

GD Graphics Library and related Perl modules

yum install rrdtool perl-rrdtool perl-GD

42 Installation

cd optpkg or path of your choice

wget httpdownloadssourceforgenetprojectnagiosgraphnagiosgraph152nagiosgraph-

152targzwget

tar xvzf nagiosgraph-152targz

cd nagiosgraph-152

installpl --check-prereq

hellip

installpl --layout standalone --prefix usrlocalnagiosgraph --nagios-perfdata-

file=varlognagiosperfdatalog --var-dir=varnagios

hellip

hellip

Continue with this configuration [y] y

hellip

Solace Nagios Toolset Integration

11

43 Post install setup Run the Solace provided Nagios that modify the Config files appropriately Edit the script and change the file paths if

necessary at the top of the script

Running setup_nagiossolace can be deferred until Solace plugins are installed (in following section)If Solace specific

tools are not in place or the Config files not setup corresponding steps will be skipped

431 Restarting Services

chkconfig httpd on ampamp chkconfig nagios on

systemctl restart httpd

systemctl restart nagios

setup_nagiossolace

Checking required files

hellip

Setting up config files

hellip

Restarting services

nagios

nagios (pid 4155) is running

httpd

hellip

Solace Nagios Toolset Integration

12

44 Verification The above script adds another host by name ldquolocalhost-grrdquo with graphics enabled that can be used to test the

Nagiosgraph install Sample page is shown below

Solace Nagios Toolset Integration

13

Clicking on the graph symbols to the right of service name brings up a graph with various time bracket A sample graph

for one of the services (HTTP) for a day is shown below

Solace Nagios Toolset Integration

14

5 Solace Nagios Plugin Setup

51 Prerequisites Solace Nagios plugin requirs the following tools packages

Perl modules to handle XML amp Nagios

Solace pluging package

yum install perl-CPAN perl-XML-LibXML perl-Nagios-Plugin

perl -MCPAN -e shell

cpangt install NagiosPlugin

cpangt install NagiosPluginDieNicely

cpangt install LWPUserAgent

cpangt install NagiosConfig

cpangt exit

52 Installation

1 Create Sample file

Create a sample router Config This file used by the setup script (below) to create commands for this config This step

can also be repeated for other routersvpns using the command mk_nagiossolacecfg script

cat cfgsolace-vmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=19216856102

HOSTPORT=8080

USERNAME=admin

PASSWORD=

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

Solace Nagios Toolset Integration

15

2 Setup Plugin

Run the setup_nagiossolace script This will setup both Nagiosgraph and the solace plugin If Nagisgraph is already

setup the related steps will be skipped

mk_nagiossolacecfg cfgsolace-vmr2cfg

Generating Nagios config for Solace appliance

Using cfgfile cfgsolace-vmr2cfg

Using template file cfgsolace-templaterouter-no_bi_bridgecfg

Generating config

etcnagiosobjectssolace-solace-vmr2cfg

Setting up config files

etcnagiosnagioscfg

Solace Nagios Toolset Integration

16

53 Post Install Setup

54 Verification If the sample router info is successfully configured a host entry matching the appliance hostname from Config file will

be added to the web page

A sample entry is shown below

Solace Nagios Toolset Integration

17

Clicking on the graph icon to the right of the service name brings up the graphs with options A sample graph is shown

below

Solace Nagios Toolset Integration

18

6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the

external command file Nagios would periodically check the external command file and process them This ldquopassive

checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured

hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status

updates and alerting

61 Prerequisites

62 Installation

1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291

(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)

2 Untar and follow install instructions in the package

$ cd optpkg (or directory of your choice)

$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site

for latest package name and replace)

$ tar xvzf nsca-291targz (adjust version accordingly)

$ cd optpkgnsca-291 (adjust version accordingly)

$ configure

Configuration summary for nsca 291 01-27-2012

General Options

-------------------------

NSCA port 5667

NSCA user nagios

NSCA group nagios

$ make all

Solace Nagios Toolset Integration

19

63 Post install setup

1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca

This will generate required Config file and setup nsca as service

NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP

setup_nsca -s optpkgnsca-291

srcdir optpkgnsca-291 ok

target dir usrlocalnsca ok

Checking

hellip

Installing

hellip

Creating service

Installation complete

2 Start service

Now that the component is installed start it up

systemctl restart nsca

Solace Nagios Toolset Integration

20

64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl

ndashh for usage info

641 Sending Host Alert

send_ngalertpl -n vmr3 -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending host command vmr30 - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0

Output - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe

[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection

Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -

[Status=UP]

Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]

Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]

Solace Nagios Toolset Integration

21

642 Sending Service Alert

send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending service command vmr3hardware0 - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service

Description hardware Return Code 0 Output - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe

Nov 9 154116 centos7-vm nsca[9888] End of connection

Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND

PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]

Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]

Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]

Solace Nagios Toolset Integration

22

7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing

monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an

active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be

configured on both the server and the client end though the client end changes are relatively light

In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server

as client

71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that

network connectivity exists between the server and client hosts and the ports are open both ways

72 Installation

721 Server (CentOS)

In addition to Nagios server packages we would need to install the following These were already covered in previous

sections and listed here for completeness and for any standalone installs

NRPE

Nagios Pluggins

OpenSSL

yum install -y nrpe nagios-plugins-all openssl

722 Client (Ubuntu)

On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component

apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-

standard

Solace Nagios Toolset Integration

23

73 Post Install Configuration

731 Client Configuration

1 Add Nagios server IP to allowed_hosts in nrpecfg

grep ^allowed_hosts etcnagiosnrpecfg

allowed_hosts=127001192168562011921681223

2 Restart service

systemctl start nagios-nrpe-server

732 Server Configuration

1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in

ldquoSetup Scripts and Config Filesrdquo section

2 Update nagioscfg (etcnagios by default) and add the client config file

For eg

rootcentos7-vm confd tail etcnagiosnagioscfg

cfg_file=etcnagiosobjectsnrpe-test1cfg

3 Now verify the config and fix any errors

nagios -v etcnagiosnagioscfg

4 Restart services

systemctl restart nagios

systemctl restart nrpe

systemctl restart httpd

Solace Nagios Toolset Integration

24

74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot

below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status

75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and

differences

1 NCSA does passive check This mean client choses when to send the status to the server This allowes

fine grain control over the update timing and useful for near real time updates

2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically

This leads to simpler install

3 NRPE does active check The server pulls the info from the client periodically This is better suited for

centrally adminsterored policies

4 NRPE requires a daemon be runnion on all the clients to service NRPE requests

5 NCSA is not in active development anymore (no updates since 2013)

Solace Nagios Toolset Integration

25

8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD

(Round Robin Database) It can collect manage and display graphs of collected data

81 Prerequisites

811 Cacti Dependencies

Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use

MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration

unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not

required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files

82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are

instructions and materials available that can be readily used Here are some lnks

httpdocscactinetmanual0881_installation1_install_unix

httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-

fedora-17-12

httpwwwcactinetdownloadsdocshtmldebugginghtml

83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to

Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for

illustration which can be replaced with any monitoring using right scripts

Solace Nagios Toolset Integration

26

831 Add Custom Data Input method

Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the

solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script

need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected

form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot

do any validation on Output Field

8311 Data Input

8312 Input

8313 Output

Solace Nagios Toolset Integration

27

832 Add Custom Data Template

Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to

declare info for the specific queue (router name vpn name queue name and credentials)

Solace Nagios Toolset Integration

28

833 Add a custom Graph Template

Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph

Template Items)

Solace Nagios Toolset Integration

29

834 Add Solace Appliance as Device

Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to

run host specific services like ping test

8341 Add Data Source List

Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown

here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the

correspoinding RRD files for these resources

Solace Nagios Toolset Integration

30

8342 Add Graph List

Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the

specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool

Solace Nagios Toolset Integration

31

84 Verification

841 Basic Install

Once you have basic Cacti setup up and running you should be able to access the install and do further configuration

thru web admin page at

httpltyour-server-ipgtcacti

After configuration you would be able to see basic system metrics for the localhost that would look similar to this

Solace Nagios Toolset Integration

32

842 Solace host check

Once Solce router is added as a device from he Devices link you would be able view the router with basic health test

such as ping status

843 Solace VPN Metrics Sample

Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue

statistics showing queue depth for a single queue on a VPN

Solace Nagios Toolset Integration

33

As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of

aggregation of queue stats over various window period from hour to year

Solace Nagios Toolset Integration

34

9 Additional Topics

91 Admin Tasks

911 Creating custom Maps

Create custom maps for any outputs that need it for example turn the iface counter values into gauge

Add to usrlocalnagiosgraphetcnagiosgraphmap

Service type iface

outputIFACE OK - [Interface161EnabledyesLinkyes]

perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-

bytes=363571175913

perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)

and push s [ if_stats

[tx-pkts COUNTER int $1 ]

[rx-pkts COUNTER int $2 ]

[tx-bytes COUNTER int $3 ]

[rx-bytes COUNTER int $4 ] ]

912 Adding additional routers VPNs

Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run

mk_nagiossolacecfg to generate required solace plugin Config files

Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be

added into etcnagiosnagioscfg file

Note that the services need to be restarted for the changes to be effective

913 Putting Nagios on Verbose mode

Change debug level in Nagios config to 2048

grep ^debug_level etcnagiosnagioscfg

debug_level=2048

Now check the debug logs All commands executed and output will be logged over there

Solace Nagios Toolset Integration

35

tail varlognagiosnagiosdebug

[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING

[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w

usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --

queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet

[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w

usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --

vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet

[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING

914 Modifying iptables

When you Cant access http from outside or when telnet to server on port 80 fail

Check Iptables routes

iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT

iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT

iptables-save | tee etcsysconfigiptables

service iptables save

service iptables restart

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

6

2 Why Solace Solace technology efficiently moves information between all kinds of applications users and devices anywhere in the

world over all kinds of networks Solace makes its state-of-the-art data movement capabilities available via hardware

and software ldquomessage routersrdquo that can meet the needs of any application or deployment environment Solacersquos

unique solution offers unmatched capacity performance robustness and TCO so our customers can focus on seizing

business opportunities instead of building and maintaining complex data distribution infrastructure

Superior Performance Solacersquos hardware and software messaging middleware products can cost-effectively meet the performance needs of

any application with feature parity and interoperability that lets companies start small and scale to support higher

volume or more demanding requirements over time and purpose-built appliances that offer 50-100x higher

performance than any other technology for customers or applications that require extremely high capacity or low

latency

Robustness Solace offers high availability (HA) and disaster recovery (DR) without the need for 3rd party products and fast failover

times no other solution can match Distributing data via dedicated TCP connections ensures an orderly well-behaved

system under load and patented techniques ensure that the performance of publishers and high-speed consumers is

never impacted by slow consumers

Simple Architecture Modern enterprises run applications that demand many kinds of data movement such as persistent messaging web

streaming WAN distribution and cloud-based communications By supporting all kinds of data movement with a unified

platform that can be deployed as a small-footprint software broker or high-capacity rack-mounted appliance Solace lets

architects design an end-to-end infrastructure thatrsquos easy to build applications for integrate with existing technologies

secure and scale

Simple Operations Solacersquos solution features a shared administration framework for all kinds of data movement deployment models and

network environments so itrsquos easy for IT staff to deploy monitor manage and upgrade their Solace-based messaging

environment

Cost Savings Solace reduces expenses with high-capacity hardware flexible software and the ability to deploy the right solution for

each problem Solacersquos support for many kinds of messaging lets you replace multiple messaging products with just

one built-in HA DR WAN and Web functionality eliminate the need for third-party products

Solace Nagios Toolset Integration

7

3 Nagios Setup

31 Prerequisites A working Nagios installatin requires the following packagestools

Httpd (Apache)

RRDTool

GD Grahics library and related Perl modules

NRPE (Nagios Remote Plugin Executor)

yum install -y httpd php gcc glibc glibc-common gd gd-devel make net-snmp wget

yum install rrdtool perl-rrdtool perl-GD

yum install perl-CGI perl-Time-HiRes

1 Enabling EPEL

The steps for enabling Extra Packge Repositor (EPEL) are release specific and please check for your install

CentOS 7

wget httpdlfedoraprojectorgpubepel7x86_64eepel-release-7-5noarchrpm

rpm -ivh epel-release-7-5noarchrpm

CentOS 63

rpm -Uvh httpdlfedoraprojectorgpubepel6x86_64epel-release-6-8noarchrpm

rpm -Uvh httprpmsfamillecolletcomenterpriseremi-release-6rpm

32 Installation

yum -y install nagios nagios-plugins-all nagios-plugins-nrpe nrpe

Solace Nagios Toolset Integration

8

33 Post install setup

331 HTTP Setup

Setting up an HTTP server (such as Apache) is beyond the scope of this document Pl refer to the documentation for

your HTTP server Nagios setup should have dropped the required config file to setup URL Aliases The file is typically

ltOptioalConfigDirgtnagioscfg (Eg etchttpdconfdnagioscfg)

If Checking Webadmin below doesnrsquot work check the httpd config and logs to see if this file was processed

332 Firewall setup

Firewall setup and making changes to existing firewall configuration is again beyond the scope of this document For

eg if you are using iptables (default firewall on CentOS) then you may need to explicitly allow incoming connection on

required ports (Refer Additional Topics Section)

333 Nagios Admin User

Create password for nagiosadmin user for web access

htpasswd ndashc etcnagiospasswd nagiosadmin

systemctl restart httpd

systemctl restart nagios

Solace Nagios Toolset Integration

9

34 Verification Point the webbrowser to httpyour-server-ip[port]nagios

Login with the username nagiosadmin and the password above step (htpassword command)

Sample page is shown below

Solace Nagios Toolset Integration

10

4 Nagiosgraph Setup

41 Prerequisites NagiosGraph installation requires the following packagestools

RRDTool and related Perl modules

GD Graphics Library and related Perl modules

yum install rrdtool perl-rrdtool perl-GD

42 Installation

cd optpkg or path of your choice

wget httpdownloadssourceforgenetprojectnagiosgraphnagiosgraph152nagiosgraph-

152targzwget

tar xvzf nagiosgraph-152targz

cd nagiosgraph-152

installpl --check-prereq

hellip

installpl --layout standalone --prefix usrlocalnagiosgraph --nagios-perfdata-

file=varlognagiosperfdatalog --var-dir=varnagios

hellip

hellip

Continue with this configuration [y] y

hellip

Solace Nagios Toolset Integration

11

43 Post install setup Run the Solace provided Nagios that modify the Config files appropriately Edit the script and change the file paths if

necessary at the top of the script

Running setup_nagiossolace can be deferred until Solace plugins are installed (in following section)If Solace specific

tools are not in place or the Config files not setup corresponding steps will be skipped

431 Restarting Services

chkconfig httpd on ampamp chkconfig nagios on

systemctl restart httpd

systemctl restart nagios

setup_nagiossolace

Checking required files

hellip

Setting up config files

hellip

Restarting services

nagios

nagios (pid 4155) is running

httpd

hellip

Solace Nagios Toolset Integration

12

44 Verification The above script adds another host by name ldquolocalhost-grrdquo with graphics enabled that can be used to test the

Nagiosgraph install Sample page is shown below

Solace Nagios Toolset Integration

13

Clicking on the graph symbols to the right of service name brings up a graph with various time bracket A sample graph

for one of the services (HTTP) for a day is shown below

Solace Nagios Toolset Integration

14

5 Solace Nagios Plugin Setup

51 Prerequisites Solace Nagios plugin requirs the following tools packages

Perl modules to handle XML amp Nagios

Solace pluging package

yum install perl-CPAN perl-XML-LibXML perl-Nagios-Plugin

perl -MCPAN -e shell

cpangt install NagiosPlugin

cpangt install NagiosPluginDieNicely

cpangt install LWPUserAgent

cpangt install NagiosConfig

cpangt exit

52 Installation

1 Create Sample file

Create a sample router Config This file used by the setup script (below) to create commands for this config This step

can also be repeated for other routersvpns using the command mk_nagiossolacecfg script

cat cfgsolace-vmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=19216856102

HOSTPORT=8080

USERNAME=admin

PASSWORD=

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

Solace Nagios Toolset Integration

15

2 Setup Plugin

Run the setup_nagiossolace script This will setup both Nagiosgraph and the solace plugin If Nagisgraph is already

setup the related steps will be skipped

mk_nagiossolacecfg cfgsolace-vmr2cfg

Generating Nagios config for Solace appliance

Using cfgfile cfgsolace-vmr2cfg

Using template file cfgsolace-templaterouter-no_bi_bridgecfg

Generating config

etcnagiosobjectssolace-solace-vmr2cfg

Setting up config files

etcnagiosnagioscfg

Solace Nagios Toolset Integration

16

53 Post Install Setup

54 Verification If the sample router info is successfully configured a host entry matching the appliance hostname from Config file will

be added to the web page

A sample entry is shown below

Solace Nagios Toolset Integration

17

Clicking on the graph icon to the right of the service name brings up the graphs with options A sample graph is shown

below

Solace Nagios Toolset Integration

18

6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the

external command file Nagios would periodically check the external command file and process them This ldquopassive

checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured

hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status

updates and alerting

61 Prerequisites

62 Installation

1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291

(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)

2 Untar and follow install instructions in the package

$ cd optpkg (or directory of your choice)

$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site

for latest package name and replace)

$ tar xvzf nsca-291targz (adjust version accordingly)

$ cd optpkgnsca-291 (adjust version accordingly)

$ configure

Configuration summary for nsca 291 01-27-2012

General Options

-------------------------

NSCA port 5667

NSCA user nagios

NSCA group nagios

$ make all

Solace Nagios Toolset Integration

19

63 Post install setup

1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca

This will generate required Config file and setup nsca as service

NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP

setup_nsca -s optpkgnsca-291

srcdir optpkgnsca-291 ok

target dir usrlocalnsca ok

Checking

hellip

Installing

hellip

Creating service

Installation complete

2 Start service

Now that the component is installed start it up

systemctl restart nsca

Solace Nagios Toolset Integration

20

64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl

ndashh for usage info

641 Sending Host Alert

send_ngalertpl -n vmr3 -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending host command vmr30 - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0

Output - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe

[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection

Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -

[Status=UP]

Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]

Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]

Solace Nagios Toolset Integration

21

642 Sending Service Alert

send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending service command vmr3hardware0 - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service

Description hardware Return Code 0 Output - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe

Nov 9 154116 centos7-vm nsca[9888] End of connection

Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND

PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]

Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]

Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]

Solace Nagios Toolset Integration

22

7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing

monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an

active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be

configured on both the server and the client end though the client end changes are relatively light

In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server

as client

71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that

network connectivity exists between the server and client hosts and the ports are open both ways

72 Installation

721 Server (CentOS)

In addition to Nagios server packages we would need to install the following These were already covered in previous

sections and listed here for completeness and for any standalone installs

NRPE

Nagios Pluggins

OpenSSL

yum install -y nrpe nagios-plugins-all openssl

722 Client (Ubuntu)

On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component

apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-

standard

Solace Nagios Toolset Integration

23

73 Post Install Configuration

731 Client Configuration

1 Add Nagios server IP to allowed_hosts in nrpecfg

grep ^allowed_hosts etcnagiosnrpecfg

allowed_hosts=127001192168562011921681223

2 Restart service

systemctl start nagios-nrpe-server

732 Server Configuration

1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in

ldquoSetup Scripts and Config Filesrdquo section

2 Update nagioscfg (etcnagios by default) and add the client config file

For eg

rootcentos7-vm confd tail etcnagiosnagioscfg

cfg_file=etcnagiosobjectsnrpe-test1cfg

3 Now verify the config and fix any errors

nagios -v etcnagiosnagioscfg

4 Restart services

systemctl restart nagios

systemctl restart nrpe

systemctl restart httpd

Solace Nagios Toolset Integration

24

74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot

below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status

75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and

differences

1 NCSA does passive check This mean client choses when to send the status to the server This allowes

fine grain control over the update timing and useful for near real time updates

2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically

This leads to simpler install

3 NRPE does active check The server pulls the info from the client periodically This is better suited for

centrally adminsterored policies

4 NRPE requires a daemon be runnion on all the clients to service NRPE requests

5 NCSA is not in active development anymore (no updates since 2013)

Solace Nagios Toolset Integration

25

8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD

(Round Robin Database) It can collect manage and display graphs of collected data

81 Prerequisites

811 Cacti Dependencies

Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use

MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration

unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not

required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files

82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are

instructions and materials available that can be readily used Here are some lnks

httpdocscactinetmanual0881_installation1_install_unix

httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-

fedora-17-12

httpwwwcactinetdownloadsdocshtmldebugginghtml

83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to

Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for

illustration which can be replaced with any monitoring using right scripts

Solace Nagios Toolset Integration

26

831 Add Custom Data Input method

Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the

solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script

need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected

form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot

do any validation on Output Field

8311 Data Input

8312 Input

8313 Output

Solace Nagios Toolset Integration

27

832 Add Custom Data Template

Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to

declare info for the specific queue (router name vpn name queue name and credentials)

Solace Nagios Toolset Integration

28

833 Add a custom Graph Template

Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph

Template Items)

Solace Nagios Toolset Integration

29

834 Add Solace Appliance as Device

Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to

run host specific services like ping test

8341 Add Data Source List

Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown

here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the

correspoinding RRD files for these resources

Solace Nagios Toolset Integration

30

8342 Add Graph List

Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the

specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool

Solace Nagios Toolset Integration

31

84 Verification

841 Basic Install

Once you have basic Cacti setup up and running you should be able to access the install and do further configuration

thru web admin page at

httpltyour-server-ipgtcacti

After configuration you would be able to see basic system metrics for the localhost that would look similar to this

Solace Nagios Toolset Integration

32

842 Solace host check

Once Solce router is added as a device from he Devices link you would be able view the router with basic health test

such as ping status

843 Solace VPN Metrics Sample

Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue

statistics showing queue depth for a single queue on a VPN

Solace Nagios Toolset Integration

33

As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of

aggregation of queue stats over various window period from hour to year

Solace Nagios Toolset Integration

34

9 Additional Topics

91 Admin Tasks

911 Creating custom Maps

Create custom maps for any outputs that need it for example turn the iface counter values into gauge

Add to usrlocalnagiosgraphetcnagiosgraphmap

Service type iface

outputIFACE OK - [Interface161EnabledyesLinkyes]

perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-

bytes=363571175913

perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)

and push s [ if_stats

[tx-pkts COUNTER int $1 ]

[rx-pkts COUNTER int $2 ]

[tx-bytes COUNTER int $3 ]

[rx-bytes COUNTER int $4 ] ]

912 Adding additional routers VPNs

Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run

mk_nagiossolacecfg to generate required solace plugin Config files

Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be

added into etcnagiosnagioscfg file

Note that the services need to be restarted for the changes to be effective

913 Putting Nagios on Verbose mode

Change debug level in Nagios config to 2048

grep ^debug_level etcnagiosnagioscfg

debug_level=2048

Now check the debug logs All commands executed and output will be logged over there

Solace Nagios Toolset Integration

35

tail varlognagiosnagiosdebug

[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING

[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w

usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --

queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet

[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w

usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --

vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet

[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING

914 Modifying iptables

When you Cant access http from outside or when telnet to server on port 80 fail

Check Iptables routes

iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT

iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT

iptables-save | tee etcsysconfigiptables

service iptables save

service iptables restart

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

7

3 Nagios Setup

31 Prerequisites A working Nagios installatin requires the following packagestools

Httpd (Apache)

RRDTool

GD Grahics library and related Perl modules

NRPE (Nagios Remote Plugin Executor)

yum install -y httpd php gcc glibc glibc-common gd gd-devel make net-snmp wget

yum install rrdtool perl-rrdtool perl-GD

yum install perl-CGI perl-Time-HiRes

1 Enabling EPEL

The steps for enabling Extra Packge Repositor (EPEL) are release specific and please check for your install

CentOS 7

wget httpdlfedoraprojectorgpubepel7x86_64eepel-release-7-5noarchrpm

rpm -ivh epel-release-7-5noarchrpm

CentOS 63

rpm -Uvh httpdlfedoraprojectorgpubepel6x86_64epel-release-6-8noarchrpm

rpm -Uvh httprpmsfamillecolletcomenterpriseremi-release-6rpm

32 Installation

yum -y install nagios nagios-plugins-all nagios-plugins-nrpe nrpe

Solace Nagios Toolset Integration

8

33 Post install setup

331 HTTP Setup

Setting up an HTTP server (such as Apache) is beyond the scope of this document Pl refer to the documentation for

your HTTP server Nagios setup should have dropped the required config file to setup URL Aliases The file is typically

ltOptioalConfigDirgtnagioscfg (Eg etchttpdconfdnagioscfg)

If Checking Webadmin below doesnrsquot work check the httpd config and logs to see if this file was processed

332 Firewall setup

Firewall setup and making changes to existing firewall configuration is again beyond the scope of this document For

eg if you are using iptables (default firewall on CentOS) then you may need to explicitly allow incoming connection on

required ports (Refer Additional Topics Section)

333 Nagios Admin User

Create password for nagiosadmin user for web access

htpasswd ndashc etcnagiospasswd nagiosadmin

systemctl restart httpd

systemctl restart nagios

Solace Nagios Toolset Integration

9

34 Verification Point the webbrowser to httpyour-server-ip[port]nagios

Login with the username nagiosadmin and the password above step (htpassword command)

Sample page is shown below

Solace Nagios Toolset Integration

10

4 Nagiosgraph Setup

41 Prerequisites NagiosGraph installation requires the following packagestools

RRDTool and related Perl modules

GD Graphics Library and related Perl modules

yum install rrdtool perl-rrdtool perl-GD

42 Installation

cd optpkg or path of your choice

wget httpdownloadssourceforgenetprojectnagiosgraphnagiosgraph152nagiosgraph-

152targzwget

tar xvzf nagiosgraph-152targz

cd nagiosgraph-152

installpl --check-prereq

hellip

installpl --layout standalone --prefix usrlocalnagiosgraph --nagios-perfdata-

file=varlognagiosperfdatalog --var-dir=varnagios

hellip

hellip

Continue with this configuration [y] y

hellip

Solace Nagios Toolset Integration

11

43 Post install setup Run the Solace provided Nagios that modify the Config files appropriately Edit the script and change the file paths if

necessary at the top of the script

Running setup_nagiossolace can be deferred until Solace plugins are installed (in following section)If Solace specific

tools are not in place or the Config files not setup corresponding steps will be skipped

431 Restarting Services

chkconfig httpd on ampamp chkconfig nagios on

systemctl restart httpd

systemctl restart nagios

setup_nagiossolace

Checking required files

hellip

Setting up config files

hellip

Restarting services

nagios

nagios (pid 4155) is running

httpd

hellip

Solace Nagios Toolset Integration

12

44 Verification The above script adds another host by name ldquolocalhost-grrdquo with graphics enabled that can be used to test the

Nagiosgraph install Sample page is shown below

Solace Nagios Toolset Integration

13

Clicking on the graph symbols to the right of service name brings up a graph with various time bracket A sample graph

for one of the services (HTTP) for a day is shown below

Solace Nagios Toolset Integration

14

5 Solace Nagios Plugin Setup

51 Prerequisites Solace Nagios plugin requirs the following tools packages

Perl modules to handle XML amp Nagios

Solace pluging package

yum install perl-CPAN perl-XML-LibXML perl-Nagios-Plugin

perl -MCPAN -e shell

cpangt install NagiosPlugin

cpangt install NagiosPluginDieNicely

cpangt install LWPUserAgent

cpangt install NagiosConfig

cpangt exit

52 Installation

1 Create Sample file

Create a sample router Config This file used by the setup script (below) to create commands for this config This step

can also be repeated for other routersvpns using the command mk_nagiossolacecfg script

cat cfgsolace-vmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=19216856102

HOSTPORT=8080

USERNAME=admin

PASSWORD=

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

Solace Nagios Toolset Integration

15

2 Setup Plugin

Run the setup_nagiossolace script This will setup both Nagiosgraph and the solace plugin If Nagisgraph is already

setup the related steps will be skipped

mk_nagiossolacecfg cfgsolace-vmr2cfg

Generating Nagios config for Solace appliance

Using cfgfile cfgsolace-vmr2cfg

Using template file cfgsolace-templaterouter-no_bi_bridgecfg

Generating config

etcnagiosobjectssolace-solace-vmr2cfg

Setting up config files

etcnagiosnagioscfg

Solace Nagios Toolset Integration

16

53 Post Install Setup

54 Verification If the sample router info is successfully configured a host entry matching the appliance hostname from Config file will

be added to the web page

A sample entry is shown below

Solace Nagios Toolset Integration

17

Clicking on the graph icon to the right of the service name brings up the graphs with options A sample graph is shown

below

Solace Nagios Toolset Integration

18

6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the

external command file Nagios would periodically check the external command file and process them This ldquopassive

checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured

hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status

updates and alerting

61 Prerequisites

62 Installation

1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291

(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)

2 Untar and follow install instructions in the package

$ cd optpkg (or directory of your choice)

$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site

for latest package name and replace)

$ tar xvzf nsca-291targz (adjust version accordingly)

$ cd optpkgnsca-291 (adjust version accordingly)

$ configure

Configuration summary for nsca 291 01-27-2012

General Options

-------------------------

NSCA port 5667

NSCA user nagios

NSCA group nagios

$ make all

Solace Nagios Toolset Integration

19

63 Post install setup

1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca

This will generate required Config file and setup nsca as service

NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP

setup_nsca -s optpkgnsca-291

srcdir optpkgnsca-291 ok

target dir usrlocalnsca ok

Checking

hellip

Installing

hellip

Creating service

Installation complete

2 Start service

Now that the component is installed start it up

systemctl restart nsca

Solace Nagios Toolset Integration

20

64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl

ndashh for usage info

641 Sending Host Alert

send_ngalertpl -n vmr3 -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending host command vmr30 - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0

Output - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe

[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection

Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -

[Status=UP]

Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]

Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]

Solace Nagios Toolset Integration

21

642 Sending Service Alert

send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending service command vmr3hardware0 - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service

Description hardware Return Code 0 Output - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe

Nov 9 154116 centos7-vm nsca[9888] End of connection

Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND

PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]

Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]

Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]

Solace Nagios Toolset Integration

22

7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing

monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an

active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be

configured on both the server and the client end though the client end changes are relatively light

In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server

as client

71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that

network connectivity exists between the server and client hosts and the ports are open both ways

72 Installation

721 Server (CentOS)

In addition to Nagios server packages we would need to install the following These were already covered in previous

sections and listed here for completeness and for any standalone installs

NRPE

Nagios Pluggins

OpenSSL

yum install -y nrpe nagios-plugins-all openssl

722 Client (Ubuntu)

On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component

apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-

standard

Solace Nagios Toolset Integration

23

73 Post Install Configuration

731 Client Configuration

1 Add Nagios server IP to allowed_hosts in nrpecfg

grep ^allowed_hosts etcnagiosnrpecfg

allowed_hosts=127001192168562011921681223

2 Restart service

systemctl start nagios-nrpe-server

732 Server Configuration

1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in

ldquoSetup Scripts and Config Filesrdquo section

2 Update nagioscfg (etcnagios by default) and add the client config file

For eg

rootcentos7-vm confd tail etcnagiosnagioscfg

cfg_file=etcnagiosobjectsnrpe-test1cfg

3 Now verify the config and fix any errors

nagios -v etcnagiosnagioscfg

4 Restart services

systemctl restart nagios

systemctl restart nrpe

systemctl restart httpd

Solace Nagios Toolset Integration

24

74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot

below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status

75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and

differences

1 NCSA does passive check This mean client choses when to send the status to the server This allowes

fine grain control over the update timing and useful for near real time updates

2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically

This leads to simpler install

3 NRPE does active check The server pulls the info from the client periodically This is better suited for

centrally adminsterored policies

4 NRPE requires a daemon be runnion on all the clients to service NRPE requests

5 NCSA is not in active development anymore (no updates since 2013)

Solace Nagios Toolset Integration

25

8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD

(Round Robin Database) It can collect manage and display graphs of collected data

81 Prerequisites

811 Cacti Dependencies

Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use

MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration

unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not

required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files

82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are

instructions and materials available that can be readily used Here are some lnks

httpdocscactinetmanual0881_installation1_install_unix

httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-

fedora-17-12

httpwwwcactinetdownloadsdocshtmldebugginghtml

83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to

Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for

illustration which can be replaced with any monitoring using right scripts

Solace Nagios Toolset Integration

26

831 Add Custom Data Input method

Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the

solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script

need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected

form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot

do any validation on Output Field

8311 Data Input

8312 Input

8313 Output

Solace Nagios Toolset Integration

27

832 Add Custom Data Template

Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to

declare info for the specific queue (router name vpn name queue name and credentials)

Solace Nagios Toolset Integration

28

833 Add a custom Graph Template

Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph

Template Items)

Solace Nagios Toolset Integration

29

834 Add Solace Appliance as Device

Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to

run host specific services like ping test

8341 Add Data Source List

Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown

here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the

correspoinding RRD files for these resources

Solace Nagios Toolset Integration

30

8342 Add Graph List

Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the

specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool

Solace Nagios Toolset Integration

31

84 Verification

841 Basic Install

Once you have basic Cacti setup up and running you should be able to access the install and do further configuration

thru web admin page at

httpltyour-server-ipgtcacti

After configuration you would be able to see basic system metrics for the localhost that would look similar to this

Solace Nagios Toolset Integration

32

842 Solace host check

Once Solce router is added as a device from he Devices link you would be able view the router with basic health test

such as ping status

843 Solace VPN Metrics Sample

Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue

statistics showing queue depth for a single queue on a VPN

Solace Nagios Toolset Integration

33

As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of

aggregation of queue stats over various window period from hour to year

Solace Nagios Toolset Integration

34

9 Additional Topics

91 Admin Tasks

911 Creating custom Maps

Create custom maps for any outputs that need it for example turn the iface counter values into gauge

Add to usrlocalnagiosgraphetcnagiosgraphmap

Service type iface

outputIFACE OK - [Interface161EnabledyesLinkyes]

perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-

bytes=363571175913

perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)

and push s [ if_stats

[tx-pkts COUNTER int $1 ]

[rx-pkts COUNTER int $2 ]

[tx-bytes COUNTER int $3 ]

[rx-bytes COUNTER int $4 ] ]

912 Adding additional routers VPNs

Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run

mk_nagiossolacecfg to generate required solace plugin Config files

Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be

added into etcnagiosnagioscfg file

Note that the services need to be restarted for the changes to be effective

913 Putting Nagios on Verbose mode

Change debug level in Nagios config to 2048

grep ^debug_level etcnagiosnagioscfg

debug_level=2048

Now check the debug logs All commands executed and output will be logged over there

Solace Nagios Toolset Integration

35

tail varlognagiosnagiosdebug

[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING

[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w

usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --

queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet

[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w

usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --

vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet

[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING

914 Modifying iptables

When you Cant access http from outside or when telnet to server on port 80 fail

Check Iptables routes

iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT

iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT

iptables-save | tee etcsysconfigiptables

service iptables save

service iptables restart

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

8

33 Post install setup

331 HTTP Setup

Setting up an HTTP server (such as Apache) is beyond the scope of this document Pl refer to the documentation for

your HTTP server Nagios setup should have dropped the required config file to setup URL Aliases The file is typically

ltOptioalConfigDirgtnagioscfg (Eg etchttpdconfdnagioscfg)

If Checking Webadmin below doesnrsquot work check the httpd config and logs to see if this file was processed

332 Firewall setup

Firewall setup and making changes to existing firewall configuration is again beyond the scope of this document For

eg if you are using iptables (default firewall on CentOS) then you may need to explicitly allow incoming connection on

required ports (Refer Additional Topics Section)

333 Nagios Admin User

Create password for nagiosadmin user for web access

htpasswd ndashc etcnagiospasswd nagiosadmin

systemctl restart httpd

systemctl restart nagios

Solace Nagios Toolset Integration

9

34 Verification Point the webbrowser to httpyour-server-ip[port]nagios

Login with the username nagiosadmin and the password above step (htpassword command)

Sample page is shown below

Solace Nagios Toolset Integration

10

4 Nagiosgraph Setup

41 Prerequisites NagiosGraph installation requires the following packagestools

RRDTool and related Perl modules

GD Graphics Library and related Perl modules

yum install rrdtool perl-rrdtool perl-GD

42 Installation

cd optpkg or path of your choice

wget httpdownloadssourceforgenetprojectnagiosgraphnagiosgraph152nagiosgraph-

152targzwget

tar xvzf nagiosgraph-152targz

cd nagiosgraph-152

installpl --check-prereq

hellip

installpl --layout standalone --prefix usrlocalnagiosgraph --nagios-perfdata-

file=varlognagiosperfdatalog --var-dir=varnagios

hellip

hellip

Continue with this configuration [y] y

hellip

Solace Nagios Toolset Integration

11

43 Post install setup Run the Solace provided Nagios that modify the Config files appropriately Edit the script and change the file paths if

necessary at the top of the script

Running setup_nagiossolace can be deferred until Solace plugins are installed (in following section)If Solace specific

tools are not in place or the Config files not setup corresponding steps will be skipped

431 Restarting Services

chkconfig httpd on ampamp chkconfig nagios on

systemctl restart httpd

systemctl restart nagios

setup_nagiossolace

Checking required files

hellip

Setting up config files

hellip

Restarting services

nagios

nagios (pid 4155) is running

httpd

hellip

Solace Nagios Toolset Integration

12

44 Verification The above script adds another host by name ldquolocalhost-grrdquo with graphics enabled that can be used to test the

Nagiosgraph install Sample page is shown below

Solace Nagios Toolset Integration

13

Clicking on the graph symbols to the right of service name brings up a graph with various time bracket A sample graph

for one of the services (HTTP) for a day is shown below

Solace Nagios Toolset Integration

14

5 Solace Nagios Plugin Setup

51 Prerequisites Solace Nagios plugin requirs the following tools packages

Perl modules to handle XML amp Nagios

Solace pluging package

yum install perl-CPAN perl-XML-LibXML perl-Nagios-Plugin

perl -MCPAN -e shell

cpangt install NagiosPlugin

cpangt install NagiosPluginDieNicely

cpangt install LWPUserAgent

cpangt install NagiosConfig

cpangt exit

52 Installation

1 Create Sample file

Create a sample router Config This file used by the setup script (below) to create commands for this config This step

can also be repeated for other routersvpns using the command mk_nagiossolacecfg script

cat cfgsolace-vmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=19216856102

HOSTPORT=8080

USERNAME=admin

PASSWORD=

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

Solace Nagios Toolset Integration

15

2 Setup Plugin

Run the setup_nagiossolace script This will setup both Nagiosgraph and the solace plugin If Nagisgraph is already

setup the related steps will be skipped

mk_nagiossolacecfg cfgsolace-vmr2cfg

Generating Nagios config for Solace appliance

Using cfgfile cfgsolace-vmr2cfg

Using template file cfgsolace-templaterouter-no_bi_bridgecfg

Generating config

etcnagiosobjectssolace-solace-vmr2cfg

Setting up config files

etcnagiosnagioscfg

Solace Nagios Toolset Integration

16

53 Post Install Setup

54 Verification If the sample router info is successfully configured a host entry matching the appliance hostname from Config file will

be added to the web page

A sample entry is shown below

Solace Nagios Toolset Integration

17

Clicking on the graph icon to the right of the service name brings up the graphs with options A sample graph is shown

below

Solace Nagios Toolset Integration

18

6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the

external command file Nagios would periodically check the external command file and process them This ldquopassive

checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured

hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status

updates and alerting

61 Prerequisites

62 Installation

1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291

(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)

2 Untar and follow install instructions in the package

$ cd optpkg (or directory of your choice)

$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site

for latest package name and replace)

$ tar xvzf nsca-291targz (adjust version accordingly)

$ cd optpkgnsca-291 (adjust version accordingly)

$ configure

Configuration summary for nsca 291 01-27-2012

General Options

-------------------------

NSCA port 5667

NSCA user nagios

NSCA group nagios

$ make all

Solace Nagios Toolset Integration

19

63 Post install setup

1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca

This will generate required Config file and setup nsca as service

NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP

setup_nsca -s optpkgnsca-291

srcdir optpkgnsca-291 ok

target dir usrlocalnsca ok

Checking

hellip

Installing

hellip

Creating service

Installation complete

2 Start service

Now that the component is installed start it up

systemctl restart nsca

Solace Nagios Toolset Integration

20

64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl

ndashh for usage info

641 Sending Host Alert

send_ngalertpl -n vmr3 -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending host command vmr30 - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0

Output - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe

[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection

Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -

[Status=UP]

Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]

Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]

Solace Nagios Toolset Integration

21

642 Sending Service Alert

send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending service command vmr3hardware0 - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service

Description hardware Return Code 0 Output - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe

Nov 9 154116 centos7-vm nsca[9888] End of connection

Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND

PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]

Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]

Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]

Solace Nagios Toolset Integration

22

7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing

monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an

active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be

configured on both the server and the client end though the client end changes are relatively light

In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server

as client

71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that

network connectivity exists between the server and client hosts and the ports are open both ways

72 Installation

721 Server (CentOS)

In addition to Nagios server packages we would need to install the following These were already covered in previous

sections and listed here for completeness and for any standalone installs

NRPE

Nagios Pluggins

OpenSSL

yum install -y nrpe nagios-plugins-all openssl

722 Client (Ubuntu)

On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component

apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-

standard

Solace Nagios Toolset Integration

23

73 Post Install Configuration

731 Client Configuration

1 Add Nagios server IP to allowed_hosts in nrpecfg

grep ^allowed_hosts etcnagiosnrpecfg

allowed_hosts=127001192168562011921681223

2 Restart service

systemctl start nagios-nrpe-server

732 Server Configuration

1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in

ldquoSetup Scripts and Config Filesrdquo section

2 Update nagioscfg (etcnagios by default) and add the client config file

For eg

rootcentos7-vm confd tail etcnagiosnagioscfg

cfg_file=etcnagiosobjectsnrpe-test1cfg

3 Now verify the config and fix any errors

nagios -v etcnagiosnagioscfg

4 Restart services

systemctl restart nagios

systemctl restart nrpe

systemctl restart httpd

Solace Nagios Toolset Integration

24

74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot

below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status

75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and

differences

1 NCSA does passive check This mean client choses when to send the status to the server This allowes

fine grain control over the update timing and useful for near real time updates

2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically

This leads to simpler install

3 NRPE does active check The server pulls the info from the client periodically This is better suited for

centrally adminsterored policies

4 NRPE requires a daemon be runnion on all the clients to service NRPE requests

5 NCSA is not in active development anymore (no updates since 2013)

Solace Nagios Toolset Integration

25

8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD

(Round Robin Database) It can collect manage and display graphs of collected data

81 Prerequisites

811 Cacti Dependencies

Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use

MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration

unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not

required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files

82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are

instructions and materials available that can be readily used Here are some lnks

httpdocscactinetmanual0881_installation1_install_unix

httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-

fedora-17-12

httpwwwcactinetdownloadsdocshtmldebugginghtml

83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to

Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for

illustration which can be replaced with any monitoring using right scripts

Solace Nagios Toolset Integration

26

831 Add Custom Data Input method

Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the

solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script

need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected

form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot

do any validation on Output Field

8311 Data Input

8312 Input

8313 Output

Solace Nagios Toolset Integration

27

832 Add Custom Data Template

Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to

declare info for the specific queue (router name vpn name queue name and credentials)

Solace Nagios Toolset Integration

28

833 Add a custom Graph Template

Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph

Template Items)

Solace Nagios Toolset Integration

29

834 Add Solace Appliance as Device

Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to

run host specific services like ping test

8341 Add Data Source List

Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown

here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the

correspoinding RRD files for these resources

Solace Nagios Toolset Integration

30

8342 Add Graph List

Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the

specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool

Solace Nagios Toolset Integration

31

84 Verification

841 Basic Install

Once you have basic Cacti setup up and running you should be able to access the install and do further configuration

thru web admin page at

httpltyour-server-ipgtcacti

After configuration you would be able to see basic system metrics for the localhost that would look similar to this

Solace Nagios Toolset Integration

32

842 Solace host check

Once Solce router is added as a device from he Devices link you would be able view the router with basic health test

such as ping status

843 Solace VPN Metrics Sample

Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue

statistics showing queue depth for a single queue on a VPN

Solace Nagios Toolset Integration

33

As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of

aggregation of queue stats over various window period from hour to year

Solace Nagios Toolset Integration

34

9 Additional Topics

91 Admin Tasks

911 Creating custom Maps

Create custom maps for any outputs that need it for example turn the iface counter values into gauge

Add to usrlocalnagiosgraphetcnagiosgraphmap

Service type iface

outputIFACE OK - [Interface161EnabledyesLinkyes]

perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-

bytes=363571175913

perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)

and push s [ if_stats

[tx-pkts COUNTER int $1 ]

[rx-pkts COUNTER int $2 ]

[tx-bytes COUNTER int $3 ]

[rx-bytes COUNTER int $4 ] ]

912 Adding additional routers VPNs

Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run

mk_nagiossolacecfg to generate required solace plugin Config files

Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be

added into etcnagiosnagioscfg file

Note that the services need to be restarted for the changes to be effective

913 Putting Nagios on Verbose mode

Change debug level in Nagios config to 2048

grep ^debug_level etcnagiosnagioscfg

debug_level=2048

Now check the debug logs All commands executed and output will be logged over there

Solace Nagios Toolset Integration

35

tail varlognagiosnagiosdebug

[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING

[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w

usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --

queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet

[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w

usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --

vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet

[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING

914 Modifying iptables

When you Cant access http from outside or when telnet to server on port 80 fail

Check Iptables routes

iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT

iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT

iptables-save | tee etcsysconfigiptables

service iptables save

service iptables restart

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

9

34 Verification Point the webbrowser to httpyour-server-ip[port]nagios

Login with the username nagiosadmin and the password above step (htpassword command)

Sample page is shown below

Solace Nagios Toolset Integration

10

4 Nagiosgraph Setup

41 Prerequisites NagiosGraph installation requires the following packagestools

RRDTool and related Perl modules

GD Graphics Library and related Perl modules

yum install rrdtool perl-rrdtool perl-GD

42 Installation

cd optpkg or path of your choice

wget httpdownloadssourceforgenetprojectnagiosgraphnagiosgraph152nagiosgraph-

152targzwget

tar xvzf nagiosgraph-152targz

cd nagiosgraph-152

installpl --check-prereq

hellip

installpl --layout standalone --prefix usrlocalnagiosgraph --nagios-perfdata-

file=varlognagiosperfdatalog --var-dir=varnagios

hellip

hellip

Continue with this configuration [y] y

hellip

Solace Nagios Toolset Integration

11

43 Post install setup Run the Solace provided Nagios that modify the Config files appropriately Edit the script and change the file paths if

necessary at the top of the script

Running setup_nagiossolace can be deferred until Solace plugins are installed (in following section)If Solace specific

tools are not in place or the Config files not setup corresponding steps will be skipped

431 Restarting Services

chkconfig httpd on ampamp chkconfig nagios on

systemctl restart httpd

systemctl restart nagios

setup_nagiossolace

Checking required files

hellip

Setting up config files

hellip

Restarting services

nagios

nagios (pid 4155) is running

httpd

hellip

Solace Nagios Toolset Integration

12

44 Verification The above script adds another host by name ldquolocalhost-grrdquo with graphics enabled that can be used to test the

Nagiosgraph install Sample page is shown below

Solace Nagios Toolset Integration

13

Clicking on the graph symbols to the right of service name brings up a graph with various time bracket A sample graph

for one of the services (HTTP) for a day is shown below

Solace Nagios Toolset Integration

14

5 Solace Nagios Plugin Setup

51 Prerequisites Solace Nagios plugin requirs the following tools packages

Perl modules to handle XML amp Nagios

Solace pluging package

yum install perl-CPAN perl-XML-LibXML perl-Nagios-Plugin

perl -MCPAN -e shell

cpangt install NagiosPlugin

cpangt install NagiosPluginDieNicely

cpangt install LWPUserAgent

cpangt install NagiosConfig

cpangt exit

52 Installation

1 Create Sample file

Create a sample router Config This file used by the setup script (below) to create commands for this config This step

can also be repeated for other routersvpns using the command mk_nagiossolacecfg script

cat cfgsolace-vmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=19216856102

HOSTPORT=8080

USERNAME=admin

PASSWORD=

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

Solace Nagios Toolset Integration

15

2 Setup Plugin

Run the setup_nagiossolace script This will setup both Nagiosgraph and the solace plugin If Nagisgraph is already

setup the related steps will be skipped

mk_nagiossolacecfg cfgsolace-vmr2cfg

Generating Nagios config for Solace appliance

Using cfgfile cfgsolace-vmr2cfg

Using template file cfgsolace-templaterouter-no_bi_bridgecfg

Generating config

etcnagiosobjectssolace-solace-vmr2cfg

Setting up config files

etcnagiosnagioscfg

Solace Nagios Toolset Integration

16

53 Post Install Setup

54 Verification If the sample router info is successfully configured a host entry matching the appliance hostname from Config file will

be added to the web page

A sample entry is shown below

Solace Nagios Toolset Integration

17

Clicking on the graph icon to the right of the service name brings up the graphs with options A sample graph is shown

below

Solace Nagios Toolset Integration

18

6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the

external command file Nagios would periodically check the external command file and process them This ldquopassive

checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured

hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status

updates and alerting

61 Prerequisites

62 Installation

1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291

(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)

2 Untar and follow install instructions in the package

$ cd optpkg (or directory of your choice)

$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site

for latest package name and replace)

$ tar xvzf nsca-291targz (adjust version accordingly)

$ cd optpkgnsca-291 (adjust version accordingly)

$ configure

Configuration summary for nsca 291 01-27-2012

General Options

-------------------------

NSCA port 5667

NSCA user nagios

NSCA group nagios

$ make all

Solace Nagios Toolset Integration

19

63 Post install setup

1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca

This will generate required Config file and setup nsca as service

NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP

setup_nsca -s optpkgnsca-291

srcdir optpkgnsca-291 ok

target dir usrlocalnsca ok

Checking

hellip

Installing

hellip

Creating service

Installation complete

2 Start service

Now that the component is installed start it up

systemctl restart nsca

Solace Nagios Toolset Integration

20

64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl

ndashh for usage info

641 Sending Host Alert

send_ngalertpl -n vmr3 -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending host command vmr30 - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0

Output - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe

[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection

Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -

[Status=UP]

Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]

Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]

Solace Nagios Toolset Integration

21

642 Sending Service Alert

send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending service command vmr3hardware0 - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service

Description hardware Return Code 0 Output - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe

Nov 9 154116 centos7-vm nsca[9888] End of connection

Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND

PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]

Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]

Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]

Solace Nagios Toolset Integration

22

7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing

monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an

active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be

configured on both the server and the client end though the client end changes are relatively light

In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server

as client

71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that

network connectivity exists between the server and client hosts and the ports are open both ways

72 Installation

721 Server (CentOS)

In addition to Nagios server packages we would need to install the following These were already covered in previous

sections and listed here for completeness and for any standalone installs

NRPE

Nagios Pluggins

OpenSSL

yum install -y nrpe nagios-plugins-all openssl

722 Client (Ubuntu)

On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component

apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-

standard

Solace Nagios Toolset Integration

23

73 Post Install Configuration

731 Client Configuration

1 Add Nagios server IP to allowed_hosts in nrpecfg

grep ^allowed_hosts etcnagiosnrpecfg

allowed_hosts=127001192168562011921681223

2 Restart service

systemctl start nagios-nrpe-server

732 Server Configuration

1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in

ldquoSetup Scripts and Config Filesrdquo section

2 Update nagioscfg (etcnagios by default) and add the client config file

For eg

rootcentos7-vm confd tail etcnagiosnagioscfg

cfg_file=etcnagiosobjectsnrpe-test1cfg

3 Now verify the config and fix any errors

nagios -v etcnagiosnagioscfg

4 Restart services

systemctl restart nagios

systemctl restart nrpe

systemctl restart httpd

Solace Nagios Toolset Integration

24

74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot

below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status

75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and

differences

1 NCSA does passive check This mean client choses when to send the status to the server This allowes

fine grain control over the update timing and useful for near real time updates

2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically

This leads to simpler install

3 NRPE does active check The server pulls the info from the client periodically This is better suited for

centrally adminsterored policies

4 NRPE requires a daemon be runnion on all the clients to service NRPE requests

5 NCSA is not in active development anymore (no updates since 2013)

Solace Nagios Toolset Integration

25

8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD

(Round Robin Database) It can collect manage and display graphs of collected data

81 Prerequisites

811 Cacti Dependencies

Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use

MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration

unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not

required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files

82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are

instructions and materials available that can be readily used Here are some lnks

httpdocscactinetmanual0881_installation1_install_unix

httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-

fedora-17-12

httpwwwcactinetdownloadsdocshtmldebugginghtml

83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to

Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for

illustration which can be replaced with any monitoring using right scripts

Solace Nagios Toolset Integration

26

831 Add Custom Data Input method

Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the

solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script

need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected

form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot

do any validation on Output Field

8311 Data Input

8312 Input

8313 Output

Solace Nagios Toolset Integration

27

832 Add Custom Data Template

Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to

declare info for the specific queue (router name vpn name queue name and credentials)

Solace Nagios Toolset Integration

28

833 Add a custom Graph Template

Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph

Template Items)

Solace Nagios Toolset Integration

29

834 Add Solace Appliance as Device

Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to

run host specific services like ping test

8341 Add Data Source List

Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown

here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the

correspoinding RRD files for these resources

Solace Nagios Toolset Integration

30

8342 Add Graph List

Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the

specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool

Solace Nagios Toolset Integration

31

84 Verification

841 Basic Install

Once you have basic Cacti setup up and running you should be able to access the install and do further configuration

thru web admin page at

httpltyour-server-ipgtcacti

After configuration you would be able to see basic system metrics for the localhost that would look similar to this

Solace Nagios Toolset Integration

32

842 Solace host check

Once Solce router is added as a device from he Devices link you would be able view the router with basic health test

such as ping status

843 Solace VPN Metrics Sample

Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue

statistics showing queue depth for a single queue on a VPN

Solace Nagios Toolset Integration

33

As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of

aggregation of queue stats over various window period from hour to year

Solace Nagios Toolset Integration

34

9 Additional Topics

91 Admin Tasks

911 Creating custom Maps

Create custom maps for any outputs that need it for example turn the iface counter values into gauge

Add to usrlocalnagiosgraphetcnagiosgraphmap

Service type iface

outputIFACE OK - [Interface161EnabledyesLinkyes]

perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-

bytes=363571175913

perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)

and push s [ if_stats

[tx-pkts COUNTER int $1 ]

[rx-pkts COUNTER int $2 ]

[tx-bytes COUNTER int $3 ]

[rx-bytes COUNTER int $4 ] ]

912 Adding additional routers VPNs

Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run

mk_nagiossolacecfg to generate required solace plugin Config files

Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be

added into etcnagiosnagioscfg file

Note that the services need to be restarted for the changes to be effective

913 Putting Nagios on Verbose mode

Change debug level in Nagios config to 2048

grep ^debug_level etcnagiosnagioscfg

debug_level=2048

Now check the debug logs All commands executed and output will be logged over there

Solace Nagios Toolset Integration

35

tail varlognagiosnagiosdebug

[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING

[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w

usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --

queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet

[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w

usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --

vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet

[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING

914 Modifying iptables

When you Cant access http from outside or when telnet to server on port 80 fail

Check Iptables routes

iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT

iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT

iptables-save | tee etcsysconfigiptables

service iptables save

service iptables restart

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

10

4 Nagiosgraph Setup

41 Prerequisites NagiosGraph installation requires the following packagestools

RRDTool and related Perl modules

GD Graphics Library and related Perl modules

yum install rrdtool perl-rrdtool perl-GD

42 Installation

cd optpkg or path of your choice

wget httpdownloadssourceforgenetprojectnagiosgraphnagiosgraph152nagiosgraph-

152targzwget

tar xvzf nagiosgraph-152targz

cd nagiosgraph-152

installpl --check-prereq

hellip

installpl --layout standalone --prefix usrlocalnagiosgraph --nagios-perfdata-

file=varlognagiosperfdatalog --var-dir=varnagios

hellip

hellip

Continue with this configuration [y] y

hellip

Solace Nagios Toolset Integration

11

43 Post install setup Run the Solace provided Nagios that modify the Config files appropriately Edit the script and change the file paths if

necessary at the top of the script

Running setup_nagiossolace can be deferred until Solace plugins are installed (in following section)If Solace specific

tools are not in place or the Config files not setup corresponding steps will be skipped

431 Restarting Services

chkconfig httpd on ampamp chkconfig nagios on

systemctl restart httpd

systemctl restart nagios

setup_nagiossolace

Checking required files

hellip

Setting up config files

hellip

Restarting services

nagios

nagios (pid 4155) is running

httpd

hellip

Solace Nagios Toolset Integration

12

44 Verification The above script adds another host by name ldquolocalhost-grrdquo with graphics enabled that can be used to test the

Nagiosgraph install Sample page is shown below

Solace Nagios Toolset Integration

13

Clicking on the graph symbols to the right of service name brings up a graph with various time bracket A sample graph

for one of the services (HTTP) for a day is shown below

Solace Nagios Toolset Integration

14

5 Solace Nagios Plugin Setup

51 Prerequisites Solace Nagios plugin requirs the following tools packages

Perl modules to handle XML amp Nagios

Solace pluging package

yum install perl-CPAN perl-XML-LibXML perl-Nagios-Plugin

perl -MCPAN -e shell

cpangt install NagiosPlugin

cpangt install NagiosPluginDieNicely

cpangt install LWPUserAgent

cpangt install NagiosConfig

cpangt exit

52 Installation

1 Create Sample file

Create a sample router Config This file used by the setup script (below) to create commands for this config This step

can also be repeated for other routersvpns using the command mk_nagiossolacecfg script

cat cfgsolace-vmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=19216856102

HOSTPORT=8080

USERNAME=admin

PASSWORD=

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

Solace Nagios Toolset Integration

15

2 Setup Plugin

Run the setup_nagiossolace script This will setup both Nagiosgraph and the solace plugin If Nagisgraph is already

setup the related steps will be skipped

mk_nagiossolacecfg cfgsolace-vmr2cfg

Generating Nagios config for Solace appliance

Using cfgfile cfgsolace-vmr2cfg

Using template file cfgsolace-templaterouter-no_bi_bridgecfg

Generating config

etcnagiosobjectssolace-solace-vmr2cfg

Setting up config files

etcnagiosnagioscfg

Solace Nagios Toolset Integration

16

53 Post Install Setup

54 Verification If the sample router info is successfully configured a host entry matching the appliance hostname from Config file will

be added to the web page

A sample entry is shown below

Solace Nagios Toolset Integration

17

Clicking on the graph icon to the right of the service name brings up the graphs with options A sample graph is shown

below

Solace Nagios Toolset Integration

18

6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the

external command file Nagios would periodically check the external command file and process them This ldquopassive

checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured

hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status

updates and alerting

61 Prerequisites

62 Installation

1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291

(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)

2 Untar and follow install instructions in the package

$ cd optpkg (or directory of your choice)

$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site

for latest package name and replace)

$ tar xvzf nsca-291targz (adjust version accordingly)

$ cd optpkgnsca-291 (adjust version accordingly)

$ configure

Configuration summary for nsca 291 01-27-2012

General Options

-------------------------

NSCA port 5667

NSCA user nagios

NSCA group nagios

$ make all

Solace Nagios Toolset Integration

19

63 Post install setup

1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca

This will generate required Config file and setup nsca as service

NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP

setup_nsca -s optpkgnsca-291

srcdir optpkgnsca-291 ok

target dir usrlocalnsca ok

Checking

hellip

Installing

hellip

Creating service

Installation complete

2 Start service

Now that the component is installed start it up

systemctl restart nsca

Solace Nagios Toolset Integration

20

64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl

ndashh for usage info

641 Sending Host Alert

send_ngalertpl -n vmr3 -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending host command vmr30 - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0

Output - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe

[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection

Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -

[Status=UP]

Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]

Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]

Solace Nagios Toolset Integration

21

642 Sending Service Alert

send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending service command vmr3hardware0 - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service

Description hardware Return Code 0 Output - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe

Nov 9 154116 centos7-vm nsca[9888] End of connection

Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND

PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]

Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]

Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]

Solace Nagios Toolset Integration

22

7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing

monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an

active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be

configured on both the server and the client end though the client end changes are relatively light

In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server

as client

71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that

network connectivity exists between the server and client hosts and the ports are open both ways

72 Installation

721 Server (CentOS)

In addition to Nagios server packages we would need to install the following These were already covered in previous

sections and listed here for completeness and for any standalone installs

NRPE

Nagios Pluggins

OpenSSL

yum install -y nrpe nagios-plugins-all openssl

722 Client (Ubuntu)

On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component

apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-

standard

Solace Nagios Toolset Integration

23

73 Post Install Configuration

731 Client Configuration

1 Add Nagios server IP to allowed_hosts in nrpecfg

grep ^allowed_hosts etcnagiosnrpecfg

allowed_hosts=127001192168562011921681223

2 Restart service

systemctl start nagios-nrpe-server

732 Server Configuration

1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in

ldquoSetup Scripts and Config Filesrdquo section

2 Update nagioscfg (etcnagios by default) and add the client config file

For eg

rootcentos7-vm confd tail etcnagiosnagioscfg

cfg_file=etcnagiosobjectsnrpe-test1cfg

3 Now verify the config and fix any errors

nagios -v etcnagiosnagioscfg

4 Restart services

systemctl restart nagios

systemctl restart nrpe

systemctl restart httpd

Solace Nagios Toolset Integration

24

74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot

below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status

75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and

differences

1 NCSA does passive check This mean client choses when to send the status to the server This allowes

fine grain control over the update timing and useful for near real time updates

2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically

This leads to simpler install

3 NRPE does active check The server pulls the info from the client periodically This is better suited for

centrally adminsterored policies

4 NRPE requires a daemon be runnion on all the clients to service NRPE requests

5 NCSA is not in active development anymore (no updates since 2013)

Solace Nagios Toolset Integration

25

8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD

(Round Robin Database) It can collect manage and display graphs of collected data

81 Prerequisites

811 Cacti Dependencies

Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use

MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration

unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not

required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files

82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are

instructions and materials available that can be readily used Here are some lnks

httpdocscactinetmanual0881_installation1_install_unix

httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-

fedora-17-12

httpwwwcactinetdownloadsdocshtmldebugginghtml

83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to

Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for

illustration which can be replaced with any monitoring using right scripts

Solace Nagios Toolset Integration

26

831 Add Custom Data Input method

Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the

solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script

need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected

form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot

do any validation on Output Field

8311 Data Input

8312 Input

8313 Output

Solace Nagios Toolset Integration

27

832 Add Custom Data Template

Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to

declare info for the specific queue (router name vpn name queue name and credentials)

Solace Nagios Toolset Integration

28

833 Add a custom Graph Template

Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph

Template Items)

Solace Nagios Toolset Integration

29

834 Add Solace Appliance as Device

Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to

run host specific services like ping test

8341 Add Data Source List

Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown

here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the

correspoinding RRD files for these resources

Solace Nagios Toolset Integration

30

8342 Add Graph List

Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the

specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool

Solace Nagios Toolset Integration

31

84 Verification

841 Basic Install

Once you have basic Cacti setup up and running you should be able to access the install and do further configuration

thru web admin page at

httpltyour-server-ipgtcacti

After configuration you would be able to see basic system metrics for the localhost that would look similar to this

Solace Nagios Toolset Integration

32

842 Solace host check

Once Solce router is added as a device from he Devices link you would be able view the router with basic health test

such as ping status

843 Solace VPN Metrics Sample

Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue

statistics showing queue depth for a single queue on a VPN

Solace Nagios Toolset Integration

33

As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of

aggregation of queue stats over various window period from hour to year

Solace Nagios Toolset Integration

34

9 Additional Topics

91 Admin Tasks

911 Creating custom Maps

Create custom maps for any outputs that need it for example turn the iface counter values into gauge

Add to usrlocalnagiosgraphetcnagiosgraphmap

Service type iface

outputIFACE OK - [Interface161EnabledyesLinkyes]

perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-

bytes=363571175913

perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)

and push s [ if_stats

[tx-pkts COUNTER int $1 ]

[rx-pkts COUNTER int $2 ]

[tx-bytes COUNTER int $3 ]

[rx-bytes COUNTER int $4 ] ]

912 Adding additional routers VPNs

Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run

mk_nagiossolacecfg to generate required solace plugin Config files

Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be

added into etcnagiosnagioscfg file

Note that the services need to be restarted for the changes to be effective

913 Putting Nagios on Verbose mode

Change debug level in Nagios config to 2048

grep ^debug_level etcnagiosnagioscfg

debug_level=2048

Now check the debug logs All commands executed and output will be logged over there

Solace Nagios Toolset Integration

35

tail varlognagiosnagiosdebug

[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING

[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w

usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --

queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet

[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w

usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --

vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet

[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING

914 Modifying iptables

When you Cant access http from outside or when telnet to server on port 80 fail

Check Iptables routes

iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT

iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT

iptables-save | tee etcsysconfigiptables

service iptables save

service iptables restart

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

11

43 Post install setup Run the Solace provided Nagios that modify the Config files appropriately Edit the script and change the file paths if

necessary at the top of the script

Running setup_nagiossolace can be deferred until Solace plugins are installed (in following section)If Solace specific

tools are not in place or the Config files not setup corresponding steps will be skipped

431 Restarting Services

chkconfig httpd on ampamp chkconfig nagios on

systemctl restart httpd

systemctl restart nagios

setup_nagiossolace

Checking required files

hellip

Setting up config files

hellip

Restarting services

nagios

nagios (pid 4155) is running

httpd

hellip

Solace Nagios Toolset Integration

12

44 Verification The above script adds another host by name ldquolocalhost-grrdquo with graphics enabled that can be used to test the

Nagiosgraph install Sample page is shown below

Solace Nagios Toolset Integration

13

Clicking on the graph symbols to the right of service name brings up a graph with various time bracket A sample graph

for one of the services (HTTP) for a day is shown below

Solace Nagios Toolset Integration

14

5 Solace Nagios Plugin Setup

51 Prerequisites Solace Nagios plugin requirs the following tools packages

Perl modules to handle XML amp Nagios

Solace pluging package

yum install perl-CPAN perl-XML-LibXML perl-Nagios-Plugin

perl -MCPAN -e shell

cpangt install NagiosPlugin

cpangt install NagiosPluginDieNicely

cpangt install LWPUserAgent

cpangt install NagiosConfig

cpangt exit

52 Installation

1 Create Sample file

Create a sample router Config This file used by the setup script (below) to create commands for this config This step

can also be repeated for other routersvpns using the command mk_nagiossolacecfg script

cat cfgsolace-vmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=19216856102

HOSTPORT=8080

USERNAME=admin

PASSWORD=

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

Solace Nagios Toolset Integration

15

2 Setup Plugin

Run the setup_nagiossolace script This will setup both Nagiosgraph and the solace plugin If Nagisgraph is already

setup the related steps will be skipped

mk_nagiossolacecfg cfgsolace-vmr2cfg

Generating Nagios config for Solace appliance

Using cfgfile cfgsolace-vmr2cfg

Using template file cfgsolace-templaterouter-no_bi_bridgecfg

Generating config

etcnagiosobjectssolace-solace-vmr2cfg

Setting up config files

etcnagiosnagioscfg

Solace Nagios Toolset Integration

16

53 Post Install Setup

54 Verification If the sample router info is successfully configured a host entry matching the appliance hostname from Config file will

be added to the web page

A sample entry is shown below

Solace Nagios Toolset Integration

17

Clicking on the graph icon to the right of the service name brings up the graphs with options A sample graph is shown

below

Solace Nagios Toolset Integration

18

6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the

external command file Nagios would periodically check the external command file and process them This ldquopassive

checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured

hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status

updates and alerting

61 Prerequisites

62 Installation

1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291

(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)

2 Untar and follow install instructions in the package

$ cd optpkg (or directory of your choice)

$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site

for latest package name and replace)

$ tar xvzf nsca-291targz (adjust version accordingly)

$ cd optpkgnsca-291 (adjust version accordingly)

$ configure

Configuration summary for nsca 291 01-27-2012

General Options

-------------------------

NSCA port 5667

NSCA user nagios

NSCA group nagios

$ make all

Solace Nagios Toolset Integration

19

63 Post install setup

1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca

This will generate required Config file and setup nsca as service

NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP

setup_nsca -s optpkgnsca-291

srcdir optpkgnsca-291 ok

target dir usrlocalnsca ok

Checking

hellip

Installing

hellip

Creating service

Installation complete

2 Start service

Now that the component is installed start it up

systemctl restart nsca

Solace Nagios Toolset Integration

20

64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl

ndashh for usage info

641 Sending Host Alert

send_ngalertpl -n vmr3 -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending host command vmr30 - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0

Output - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe

[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection

Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -

[Status=UP]

Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]

Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]

Solace Nagios Toolset Integration

21

642 Sending Service Alert

send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending service command vmr3hardware0 - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service

Description hardware Return Code 0 Output - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe

Nov 9 154116 centos7-vm nsca[9888] End of connection

Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND

PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]

Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]

Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]

Solace Nagios Toolset Integration

22

7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing

monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an

active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be

configured on both the server and the client end though the client end changes are relatively light

In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server

as client

71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that

network connectivity exists between the server and client hosts and the ports are open both ways

72 Installation

721 Server (CentOS)

In addition to Nagios server packages we would need to install the following These were already covered in previous

sections and listed here for completeness and for any standalone installs

NRPE

Nagios Pluggins

OpenSSL

yum install -y nrpe nagios-plugins-all openssl

722 Client (Ubuntu)

On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component

apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-

standard

Solace Nagios Toolset Integration

23

73 Post Install Configuration

731 Client Configuration

1 Add Nagios server IP to allowed_hosts in nrpecfg

grep ^allowed_hosts etcnagiosnrpecfg

allowed_hosts=127001192168562011921681223

2 Restart service

systemctl start nagios-nrpe-server

732 Server Configuration

1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in

ldquoSetup Scripts and Config Filesrdquo section

2 Update nagioscfg (etcnagios by default) and add the client config file

For eg

rootcentos7-vm confd tail etcnagiosnagioscfg

cfg_file=etcnagiosobjectsnrpe-test1cfg

3 Now verify the config and fix any errors

nagios -v etcnagiosnagioscfg

4 Restart services

systemctl restart nagios

systemctl restart nrpe

systemctl restart httpd

Solace Nagios Toolset Integration

24

74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot

below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status

75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and

differences

1 NCSA does passive check This mean client choses when to send the status to the server This allowes

fine grain control over the update timing and useful for near real time updates

2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically

This leads to simpler install

3 NRPE does active check The server pulls the info from the client periodically This is better suited for

centrally adminsterored policies

4 NRPE requires a daemon be runnion on all the clients to service NRPE requests

5 NCSA is not in active development anymore (no updates since 2013)

Solace Nagios Toolset Integration

25

8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD

(Round Robin Database) It can collect manage and display graphs of collected data

81 Prerequisites

811 Cacti Dependencies

Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use

MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration

unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not

required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files

82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are

instructions and materials available that can be readily used Here are some lnks

httpdocscactinetmanual0881_installation1_install_unix

httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-

fedora-17-12

httpwwwcactinetdownloadsdocshtmldebugginghtml

83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to

Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for

illustration which can be replaced with any monitoring using right scripts

Solace Nagios Toolset Integration

26

831 Add Custom Data Input method

Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the

solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script

need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected

form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot

do any validation on Output Field

8311 Data Input

8312 Input

8313 Output

Solace Nagios Toolset Integration

27

832 Add Custom Data Template

Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to

declare info for the specific queue (router name vpn name queue name and credentials)

Solace Nagios Toolset Integration

28

833 Add a custom Graph Template

Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph

Template Items)

Solace Nagios Toolset Integration

29

834 Add Solace Appliance as Device

Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to

run host specific services like ping test

8341 Add Data Source List

Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown

here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the

correspoinding RRD files for these resources

Solace Nagios Toolset Integration

30

8342 Add Graph List

Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the

specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool

Solace Nagios Toolset Integration

31

84 Verification

841 Basic Install

Once you have basic Cacti setup up and running you should be able to access the install and do further configuration

thru web admin page at

httpltyour-server-ipgtcacti

After configuration you would be able to see basic system metrics for the localhost that would look similar to this

Solace Nagios Toolset Integration

32

842 Solace host check

Once Solce router is added as a device from he Devices link you would be able view the router with basic health test

such as ping status

843 Solace VPN Metrics Sample

Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue

statistics showing queue depth for a single queue on a VPN

Solace Nagios Toolset Integration

33

As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of

aggregation of queue stats over various window period from hour to year

Solace Nagios Toolset Integration

34

9 Additional Topics

91 Admin Tasks

911 Creating custom Maps

Create custom maps for any outputs that need it for example turn the iface counter values into gauge

Add to usrlocalnagiosgraphetcnagiosgraphmap

Service type iface

outputIFACE OK - [Interface161EnabledyesLinkyes]

perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-

bytes=363571175913

perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)

and push s [ if_stats

[tx-pkts COUNTER int $1 ]

[rx-pkts COUNTER int $2 ]

[tx-bytes COUNTER int $3 ]

[rx-bytes COUNTER int $4 ] ]

912 Adding additional routers VPNs

Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run

mk_nagiossolacecfg to generate required solace plugin Config files

Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be

added into etcnagiosnagioscfg file

Note that the services need to be restarted for the changes to be effective

913 Putting Nagios on Verbose mode

Change debug level in Nagios config to 2048

grep ^debug_level etcnagiosnagioscfg

debug_level=2048

Now check the debug logs All commands executed and output will be logged over there

Solace Nagios Toolset Integration

35

tail varlognagiosnagiosdebug

[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING

[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w

usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --

queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet

[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w

usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --

vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet

[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING

914 Modifying iptables

When you Cant access http from outside or when telnet to server on port 80 fail

Check Iptables routes

iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT

iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT

iptables-save | tee etcsysconfigiptables

service iptables save

service iptables restart

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

12

44 Verification The above script adds another host by name ldquolocalhost-grrdquo with graphics enabled that can be used to test the

Nagiosgraph install Sample page is shown below

Solace Nagios Toolset Integration

13

Clicking on the graph symbols to the right of service name brings up a graph with various time bracket A sample graph

for one of the services (HTTP) for a day is shown below

Solace Nagios Toolset Integration

14

5 Solace Nagios Plugin Setup

51 Prerequisites Solace Nagios plugin requirs the following tools packages

Perl modules to handle XML amp Nagios

Solace pluging package

yum install perl-CPAN perl-XML-LibXML perl-Nagios-Plugin

perl -MCPAN -e shell

cpangt install NagiosPlugin

cpangt install NagiosPluginDieNicely

cpangt install LWPUserAgent

cpangt install NagiosConfig

cpangt exit

52 Installation

1 Create Sample file

Create a sample router Config This file used by the setup script (below) to create commands for this config This step

can also be repeated for other routersvpns using the command mk_nagiossolacecfg script

cat cfgsolace-vmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=19216856102

HOSTPORT=8080

USERNAME=admin

PASSWORD=

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

Solace Nagios Toolset Integration

15

2 Setup Plugin

Run the setup_nagiossolace script This will setup both Nagiosgraph and the solace plugin If Nagisgraph is already

setup the related steps will be skipped

mk_nagiossolacecfg cfgsolace-vmr2cfg

Generating Nagios config for Solace appliance

Using cfgfile cfgsolace-vmr2cfg

Using template file cfgsolace-templaterouter-no_bi_bridgecfg

Generating config

etcnagiosobjectssolace-solace-vmr2cfg

Setting up config files

etcnagiosnagioscfg

Solace Nagios Toolset Integration

16

53 Post Install Setup

54 Verification If the sample router info is successfully configured a host entry matching the appliance hostname from Config file will

be added to the web page

A sample entry is shown below

Solace Nagios Toolset Integration

17

Clicking on the graph icon to the right of the service name brings up the graphs with options A sample graph is shown

below

Solace Nagios Toolset Integration

18

6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the

external command file Nagios would periodically check the external command file and process them This ldquopassive

checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured

hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status

updates and alerting

61 Prerequisites

62 Installation

1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291

(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)

2 Untar and follow install instructions in the package

$ cd optpkg (or directory of your choice)

$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site

for latest package name and replace)

$ tar xvzf nsca-291targz (adjust version accordingly)

$ cd optpkgnsca-291 (adjust version accordingly)

$ configure

Configuration summary for nsca 291 01-27-2012

General Options

-------------------------

NSCA port 5667

NSCA user nagios

NSCA group nagios

$ make all

Solace Nagios Toolset Integration

19

63 Post install setup

1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca

This will generate required Config file and setup nsca as service

NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP

setup_nsca -s optpkgnsca-291

srcdir optpkgnsca-291 ok

target dir usrlocalnsca ok

Checking

hellip

Installing

hellip

Creating service

Installation complete

2 Start service

Now that the component is installed start it up

systemctl restart nsca

Solace Nagios Toolset Integration

20

64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl

ndashh for usage info

641 Sending Host Alert

send_ngalertpl -n vmr3 -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending host command vmr30 - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0

Output - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe

[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection

Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -

[Status=UP]

Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]

Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]

Solace Nagios Toolset Integration

21

642 Sending Service Alert

send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending service command vmr3hardware0 - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service

Description hardware Return Code 0 Output - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe

Nov 9 154116 centos7-vm nsca[9888] End of connection

Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND

PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]

Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]

Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]

Solace Nagios Toolset Integration

22

7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing

monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an

active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be

configured on both the server and the client end though the client end changes are relatively light

In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server

as client

71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that

network connectivity exists between the server and client hosts and the ports are open both ways

72 Installation

721 Server (CentOS)

In addition to Nagios server packages we would need to install the following These were already covered in previous

sections and listed here for completeness and for any standalone installs

NRPE

Nagios Pluggins

OpenSSL

yum install -y nrpe nagios-plugins-all openssl

722 Client (Ubuntu)

On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component

apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-

standard

Solace Nagios Toolset Integration

23

73 Post Install Configuration

731 Client Configuration

1 Add Nagios server IP to allowed_hosts in nrpecfg

grep ^allowed_hosts etcnagiosnrpecfg

allowed_hosts=127001192168562011921681223

2 Restart service

systemctl start nagios-nrpe-server

732 Server Configuration

1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in

ldquoSetup Scripts and Config Filesrdquo section

2 Update nagioscfg (etcnagios by default) and add the client config file

For eg

rootcentos7-vm confd tail etcnagiosnagioscfg

cfg_file=etcnagiosobjectsnrpe-test1cfg

3 Now verify the config and fix any errors

nagios -v etcnagiosnagioscfg

4 Restart services

systemctl restart nagios

systemctl restart nrpe

systemctl restart httpd

Solace Nagios Toolset Integration

24

74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot

below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status

75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and

differences

1 NCSA does passive check This mean client choses when to send the status to the server This allowes

fine grain control over the update timing and useful for near real time updates

2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically

This leads to simpler install

3 NRPE does active check The server pulls the info from the client periodically This is better suited for

centrally adminsterored policies

4 NRPE requires a daemon be runnion on all the clients to service NRPE requests

5 NCSA is not in active development anymore (no updates since 2013)

Solace Nagios Toolset Integration

25

8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD

(Round Robin Database) It can collect manage and display graphs of collected data

81 Prerequisites

811 Cacti Dependencies

Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use

MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration

unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not

required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files

82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are

instructions and materials available that can be readily used Here are some lnks

httpdocscactinetmanual0881_installation1_install_unix

httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-

fedora-17-12

httpwwwcactinetdownloadsdocshtmldebugginghtml

83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to

Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for

illustration which can be replaced with any monitoring using right scripts

Solace Nagios Toolset Integration

26

831 Add Custom Data Input method

Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the

solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script

need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected

form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot

do any validation on Output Field

8311 Data Input

8312 Input

8313 Output

Solace Nagios Toolset Integration

27

832 Add Custom Data Template

Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to

declare info for the specific queue (router name vpn name queue name and credentials)

Solace Nagios Toolset Integration

28

833 Add a custom Graph Template

Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph

Template Items)

Solace Nagios Toolset Integration

29

834 Add Solace Appliance as Device

Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to

run host specific services like ping test

8341 Add Data Source List

Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown

here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the

correspoinding RRD files for these resources

Solace Nagios Toolset Integration

30

8342 Add Graph List

Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the

specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool

Solace Nagios Toolset Integration

31

84 Verification

841 Basic Install

Once you have basic Cacti setup up and running you should be able to access the install and do further configuration

thru web admin page at

httpltyour-server-ipgtcacti

After configuration you would be able to see basic system metrics for the localhost that would look similar to this

Solace Nagios Toolset Integration

32

842 Solace host check

Once Solce router is added as a device from he Devices link you would be able view the router with basic health test

such as ping status

843 Solace VPN Metrics Sample

Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue

statistics showing queue depth for a single queue on a VPN

Solace Nagios Toolset Integration

33

As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of

aggregation of queue stats over various window period from hour to year

Solace Nagios Toolset Integration

34

9 Additional Topics

91 Admin Tasks

911 Creating custom Maps

Create custom maps for any outputs that need it for example turn the iface counter values into gauge

Add to usrlocalnagiosgraphetcnagiosgraphmap

Service type iface

outputIFACE OK - [Interface161EnabledyesLinkyes]

perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-

bytes=363571175913

perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)

and push s [ if_stats

[tx-pkts COUNTER int $1 ]

[rx-pkts COUNTER int $2 ]

[tx-bytes COUNTER int $3 ]

[rx-bytes COUNTER int $4 ] ]

912 Adding additional routers VPNs

Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run

mk_nagiossolacecfg to generate required solace plugin Config files

Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be

added into etcnagiosnagioscfg file

Note that the services need to be restarted for the changes to be effective

913 Putting Nagios on Verbose mode

Change debug level in Nagios config to 2048

grep ^debug_level etcnagiosnagioscfg

debug_level=2048

Now check the debug logs All commands executed and output will be logged over there

Solace Nagios Toolset Integration

35

tail varlognagiosnagiosdebug

[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING

[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w

usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --

queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet

[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w

usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --

vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet

[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING

914 Modifying iptables

When you Cant access http from outside or when telnet to server on port 80 fail

Check Iptables routes

iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT

iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT

iptables-save | tee etcsysconfigiptables

service iptables save

service iptables restart

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

13

Clicking on the graph symbols to the right of service name brings up a graph with various time bracket A sample graph

for one of the services (HTTP) for a day is shown below

Solace Nagios Toolset Integration

14

5 Solace Nagios Plugin Setup

51 Prerequisites Solace Nagios plugin requirs the following tools packages

Perl modules to handle XML amp Nagios

Solace pluging package

yum install perl-CPAN perl-XML-LibXML perl-Nagios-Plugin

perl -MCPAN -e shell

cpangt install NagiosPlugin

cpangt install NagiosPluginDieNicely

cpangt install LWPUserAgent

cpangt install NagiosConfig

cpangt exit

52 Installation

1 Create Sample file

Create a sample router Config This file used by the setup script (below) to create commands for this config This step

can also be repeated for other routersvpns using the command mk_nagiossolacecfg script

cat cfgsolace-vmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=19216856102

HOSTPORT=8080

USERNAME=admin

PASSWORD=

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

Solace Nagios Toolset Integration

15

2 Setup Plugin

Run the setup_nagiossolace script This will setup both Nagiosgraph and the solace plugin If Nagisgraph is already

setup the related steps will be skipped

mk_nagiossolacecfg cfgsolace-vmr2cfg

Generating Nagios config for Solace appliance

Using cfgfile cfgsolace-vmr2cfg

Using template file cfgsolace-templaterouter-no_bi_bridgecfg

Generating config

etcnagiosobjectssolace-solace-vmr2cfg

Setting up config files

etcnagiosnagioscfg

Solace Nagios Toolset Integration

16

53 Post Install Setup

54 Verification If the sample router info is successfully configured a host entry matching the appliance hostname from Config file will

be added to the web page

A sample entry is shown below

Solace Nagios Toolset Integration

17

Clicking on the graph icon to the right of the service name brings up the graphs with options A sample graph is shown

below

Solace Nagios Toolset Integration

18

6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the

external command file Nagios would periodically check the external command file and process them This ldquopassive

checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured

hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status

updates and alerting

61 Prerequisites

62 Installation

1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291

(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)

2 Untar and follow install instructions in the package

$ cd optpkg (or directory of your choice)

$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site

for latest package name and replace)

$ tar xvzf nsca-291targz (adjust version accordingly)

$ cd optpkgnsca-291 (adjust version accordingly)

$ configure

Configuration summary for nsca 291 01-27-2012

General Options

-------------------------

NSCA port 5667

NSCA user nagios

NSCA group nagios

$ make all

Solace Nagios Toolset Integration

19

63 Post install setup

1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca

This will generate required Config file and setup nsca as service

NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP

setup_nsca -s optpkgnsca-291

srcdir optpkgnsca-291 ok

target dir usrlocalnsca ok

Checking

hellip

Installing

hellip

Creating service

Installation complete

2 Start service

Now that the component is installed start it up

systemctl restart nsca

Solace Nagios Toolset Integration

20

64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl

ndashh for usage info

641 Sending Host Alert

send_ngalertpl -n vmr3 -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending host command vmr30 - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0

Output - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe

[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection

Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -

[Status=UP]

Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]

Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]

Solace Nagios Toolset Integration

21

642 Sending Service Alert

send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending service command vmr3hardware0 - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service

Description hardware Return Code 0 Output - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe

Nov 9 154116 centos7-vm nsca[9888] End of connection

Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND

PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]

Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]

Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]

Solace Nagios Toolset Integration

22

7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing

monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an

active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be

configured on both the server and the client end though the client end changes are relatively light

In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server

as client

71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that

network connectivity exists between the server and client hosts and the ports are open both ways

72 Installation

721 Server (CentOS)

In addition to Nagios server packages we would need to install the following These were already covered in previous

sections and listed here for completeness and for any standalone installs

NRPE

Nagios Pluggins

OpenSSL

yum install -y nrpe nagios-plugins-all openssl

722 Client (Ubuntu)

On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component

apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-

standard

Solace Nagios Toolset Integration

23

73 Post Install Configuration

731 Client Configuration

1 Add Nagios server IP to allowed_hosts in nrpecfg

grep ^allowed_hosts etcnagiosnrpecfg

allowed_hosts=127001192168562011921681223

2 Restart service

systemctl start nagios-nrpe-server

732 Server Configuration

1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in

ldquoSetup Scripts and Config Filesrdquo section

2 Update nagioscfg (etcnagios by default) and add the client config file

For eg

rootcentos7-vm confd tail etcnagiosnagioscfg

cfg_file=etcnagiosobjectsnrpe-test1cfg

3 Now verify the config and fix any errors

nagios -v etcnagiosnagioscfg

4 Restart services

systemctl restart nagios

systemctl restart nrpe

systemctl restart httpd

Solace Nagios Toolset Integration

24

74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot

below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status

75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and

differences

1 NCSA does passive check This mean client choses when to send the status to the server This allowes

fine grain control over the update timing and useful for near real time updates

2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically

This leads to simpler install

3 NRPE does active check The server pulls the info from the client periodically This is better suited for

centrally adminsterored policies

4 NRPE requires a daemon be runnion on all the clients to service NRPE requests

5 NCSA is not in active development anymore (no updates since 2013)

Solace Nagios Toolset Integration

25

8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD

(Round Robin Database) It can collect manage and display graphs of collected data

81 Prerequisites

811 Cacti Dependencies

Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use

MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration

unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not

required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files

82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are

instructions and materials available that can be readily used Here are some lnks

httpdocscactinetmanual0881_installation1_install_unix

httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-

fedora-17-12

httpwwwcactinetdownloadsdocshtmldebugginghtml

83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to

Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for

illustration which can be replaced with any monitoring using right scripts

Solace Nagios Toolset Integration

26

831 Add Custom Data Input method

Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the

solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script

need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected

form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot

do any validation on Output Field

8311 Data Input

8312 Input

8313 Output

Solace Nagios Toolset Integration

27

832 Add Custom Data Template

Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to

declare info for the specific queue (router name vpn name queue name and credentials)

Solace Nagios Toolset Integration

28

833 Add a custom Graph Template

Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph

Template Items)

Solace Nagios Toolset Integration

29

834 Add Solace Appliance as Device

Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to

run host specific services like ping test

8341 Add Data Source List

Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown

here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the

correspoinding RRD files for these resources

Solace Nagios Toolset Integration

30

8342 Add Graph List

Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the

specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool

Solace Nagios Toolset Integration

31

84 Verification

841 Basic Install

Once you have basic Cacti setup up and running you should be able to access the install and do further configuration

thru web admin page at

httpltyour-server-ipgtcacti

After configuration you would be able to see basic system metrics for the localhost that would look similar to this

Solace Nagios Toolset Integration

32

842 Solace host check

Once Solce router is added as a device from he Devices link you would be able view the router with basic health test

such as ping status

843 Solace VPN Metrics Sample

Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue

statistics showing queue depth for a single queue on a VPN

Solace Nagios Toolset Integration

33

As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of

aggregation of queue stats over various window period from hour to year

Solace Nagios Toolset Integration

34

9 Additional Topics

91 Admin Tasks

911 Creating custom Maps

Create custom maps for any outputs that need it for example turn the iface counter values into gauge

Add to usrlocalnagiosgraphetcnagiosgraphmap

Service type iface

outputIFACE OK - [Interface161EnabledyesLinkyes]

perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-

bytes=363571175913

perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)

and push s [ if_stats

[tx-pkts COUNTER int $1 ]

[rx-pkts COUNTER int $2 ]

[tx-bytes COUNTER int $3 ]

[rx-bytes COUNTER int $4 ] ]

912 Adding additional routers VPNs

Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run

mk_nagiossolacecfg to generate required solace plugin Config files

Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be

added into etcnagiosnagioscfg file

Note that the services need to be restarted for the changes to be effective

913 Putting Nagios on Verbose mode

Change debug level in Nagios config to 2048

grep ^debug_level etcnagiosnagioscfg

debug_level=2048

Now check the debug logs All commands executed and output will be logged over there

Solace Nagios Toolset Integration

35

tail varlognagiosnagiosdebug

[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING

[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w

usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --

queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet

[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w

usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --

vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet

[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING

914 Modifying iptables

When you Cant access http from outside or when telnet to server on port 80 fail

Check Iptables routes

iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT

iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT

iptables-save | tee etcsysconfigiptables

service iptables save

service iptables restart

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

14

5 Solace Nagios Plugin Setup

51 Prerequisites Solace Nagios plugin requirs the following tools packages

Perl modules to handle XML amp Nagios

Solace pluging package

yum install perl-CPAN perl-XML-LibXML perl-Nagios-Plugin

perl -MCPAN -e shell

cpangt install NagiosPlugin

cpangt install NagiosPluginDieNicely

cpangt install LWPUserAgent

cpangt install NagiosConfig

cpangt exit

52 Installation

1 Create Sample file

Create a sample router Config This file used by the setup script (below) to create commands for this config This step

can also be repeated for other routersvpns using the command mk_nagiossolacecfg script

cat cfgsolace-vmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=19216856102

HOSTPORT=8080

USERNAME=admin

PASSWORD=

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

Solace Nagios Toolset Integration

15

2 Setup Plugin

Run the setup_nagiossolace script This will setup both Nagiosgraph and the solace plugin If Nagisgraph is already

setup the related steps will be skipped

mk_nagiossolacecfg cfgsolace-vmr2cfg

Generating Nagios config for Solace appliance

Using cfgfile cfgsolace-vmr2cfg

Using template file cfgsolace-templaterouter-no_bi_bridgecfg

Generating config

etcnagiosobjectssolace-solace-vmr2cfg

Setting up config files

etcnagiosnagioscfg

Solace Nagios Toolset Integration

16

53 Post Install Setup

54 Verification If the sample router info is successfully configured a host entry matching the appliance hostname from Config file will

be added to the web page

A sample entry is shown below

Solace Nagios Toolset Integration

17

Clicking on the graph icon to the right of the service name brings up the graphs with options A sample graph is shown

below

Solace Nagios Toolset Integration

18

6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the

external command file Nagios would periodically check the external command file and process them This ldquopassive

checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured

hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status

updates and alerting

61 Prerequisites

62 Installation

1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291

(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)

2 Untar and follow install instructions in the package

$ cd optpkg (or directory of your choice)

$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site

for latest package name and replace)

$ tar xvzf nsca-291targz (adjust version accordingly)

$ cd optpkgnsca-291 (adjust version accordingly)

$ configure

Configuration summary for nsca 291 01-27-2012

General Options

-------------------------

NSCA port 5667

NSCA user nagios

NSCA group nagios

$ make all

Solace Nagios Toolset Integration

19

63 Post install setup

1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca

This will generate required Config file and setup nsca as service

NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP

setup_nsca -s optpkgnsca-291

srcdir optpkgnsca-291 ok

target dir usrlocalnsca ok

Checking

hellip

Installing

hellip

Creating service

Installation complete

2 Start service

Now that the component is installed start it up

systemctl restart nsca

Solace Nagios Toolset Integration

20

64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl

ndashh for usage info

641 Sending Host Alert

send_ngalertpl -n vmr3 -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending host command vmr30 - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0

Output - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe

[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection

Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -

[Status=UP]

Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]

Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]

Solace Nagios Toolset Integration

21

642 Sending Service Alert

send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending service command vmr3hardware0 - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service

Description hardware Return Code 0 Output - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe

Nov 9 154116 centos7-vm nsca[9888] End of connection

Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND

PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]

Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]

Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]

Solace Nagios Toolset Integration

22

7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing

monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an

active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be

configured on both the server and the client end though the client end changes are relatively light

In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server

as client

71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that

network connectivity exists between the server and client hosts and the ports are open both ways

72 Installation

721 Server (CentOS)

In addition to Nagios server packages we would need to install the following These were already covered in previous

sections and listed here for completeness and for any standalone installs

NRPE

Nagios Pluggins

OpenSSL

yum install -y nrpe nagios-plugins-all openssl

722 Client (Ubuntu)

On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component

apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-

standard

Solace Nagios Toolset Integration

23

73 Post Install Configuration

731 Client Configuration

1 Add Nagios server IP to allowed_hosts in nrpecfg

grep ^allowed_hosts etcnagiosnrpecfg

allowed_hosts=127001192168562011921681223

2 Restart service

systemctl start nagios-nrpe-server

732 Server Configuration

1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in

ldquoSetup Scripts and Config Filesrdquo section

2 Update nagioscfg (etcnagios by default) and add the client config file

For eg

rootcentos7-vm confd tail etcnagiosnagioscfg

cfg_file=etcnagiosobjectsnrpe-test1cfg

3 Now verify the config and fix any errors

nagios -v etcnagiosnagioscfg

4 Restart services

systemctl restart nagios

systemctl restart nrpe

systemctl restart httpd

Solace Nagios Toolset Integration

24

74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot

below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status

75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and

differences

1 NCSA does passive check This mean client choses when to send the status to the server This allowes

fine grain control over the update timing and useful for near real time updates

2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically

This leads to simpler install

3 NRPE does active check The server pulls the info from the client periodically This is better suited for

centrally adminsterored policies

4 NRPE requires a daemon be runnion on all the clients to service NRPE requests

5 NCSA is not in active development anymore (no updates since 2013)

Solace Nagios Toolset Integration

25

8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD

(Round Robin Database) It can collect manage and display graphs of collected data

81 Prerequisites

811 Cacti Dependencies

Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use

MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration

unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not

required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files

82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are

instructions and materials available that can be readily used Here are some lnks

httpdocscactinetmanual0881_installation1_install_unix

httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-

fedora-17-12

httpwwwcactinetdownloadsdocshtmldebugginghtml

83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to

Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for

illustration which can be replaced with any monitoring using right scripts

Solace Nagios Toolset Integration

26

831 Add Custom Data Input method

Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the

solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script

need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected

form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot

do any validation on Output Field

8311 Data Input

8312 Input

8313 Output

Solace Nagios Toolset Integration

27

832 Add Custom Data Template

Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to

declare info for the specific queue (router name vpn name queue name and credentials)

Solace Nagios Toolset Integration

28

833 Add a custom Graph Template

Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph

Template Items)

Solace Nagios Toolset Integration

29

834 Add Solace Appliance as Device

Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to

run host specific services like ping test

8341 Add Data Source List

Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown

here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the

correspoinding RRD files for these resources

Solace Nagios Toolset Integration

30

8342 Add Graph List

Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the

specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool

Solace Nagios Toolset Integration

31

84 Verification

841 Basic Install

Once you have basic Cacti setup up and running you should be able to access the install and do further configuration

thru web admin page at

httpltyour-server-ipgtcacti

After configuration you would be able to see basic system metrics for the localhost that would look similar to this

Solace Nagios Toolset Integration

32

842 Solace host check

Once Solce router is added as a device from he Devices link you would be able view the router with basic health test

such as ping status

843 Solace VPN Metrics Sample

Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue

statistics showing queue depth for a single queue on a VPN

Solace Nagios Toolset Integration

33

As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of

aggregation of queue stats over various window period from hour to year

Solace Nagios Toolset Integration

34

9 Additional Topics

91 Admin Tasks

911 Creating custom Maps

Create custom maps for any outputs that need it for example turn the iface counter values into gauge

Add to usrlocalnagiosgraphetcnagiosgraphmap

Service type iface

outputIFACE OK - [Interface161EnabledyesLinkyes]

perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-

bytes=363571175913

perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)

and push s [ if_stats

[tx-pkts COUNTER int $1 ]

[rx-pkts COUNTER int $2 ]

[tx-bytes COUNTER int $3 ]

[rx-bytes COUNTER int $4 ] ]

912 Adding additional routers VPNs

Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run

mk_nagiossolacecfg to generate required solace plugin Config files

Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be

added into etcnagiosnagioscfg file

Note that the services need to be restarted for the changes to be effective

913 Putting Nagios on Verbose mode

Change debug level in Nagios config to 2048

grep ^debug_level etcnagiosnagioscfg

debug_level=2048

Now check the debug logs All commands executed and output will be logged over there

Solace Nagios Toolset Integration

35

tail varlognagiosnagiosdebug

[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING

[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w

usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --

queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet

[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w

usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --

vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet

[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING

914 Modifying iptables

When you Cant access http from outside or when telnet to server on port 80 fail

Check Iptables routes

iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT

iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT

iptables-save | tee etcsysconfigiptables

service iptables save

service iptables restart

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

15

2 Setup Plugin

Run the setup_nagiossolace script This will setup both Nagiosgraph and the solace plugin If Nagisgraph is already

setup the related steps will be skipped

mk_nagiossolacecfg cfgsolace-vmr2cfg

Generating Nagios config for Solace appliance

Using cfgfile cfgsolace-vmr2cfg

Using template file cfgsolace-templaterouter-no_bi_bridgecfg

Generating config

etcnagiosobjectssolace-solace-vmr2cfg

Setting up config files

etcnagiosnagioscfg

Solace Nagios Toolset Integration

16

53 Post Install Setup

54 Verification If the sample router info is successfully configured a host entry matching the appliance hostname from Config file will

be added to the web page

A sample entry is shown below

Solace Nagios Toolset Integration

17

Clicking on the graph icon to the right of the service name brings up the graphs with options A sample graph is shown

below

Solace Nagios Toolset Integration

18

6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the

external command file Nagios would periodically check the external command file and process them This ldquopassive

checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured

hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status

updates and alerting

61 Prerequisites

62 Installation

1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291

(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)

2 Untar and follow install instructions in the package

$ cd optpkg (or directory of your choice)

$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site

for latest package name and replace)

$ tar xvzf nsca-291targz (adjust version accordingly)

$ cd optpkgnsca-291 (adjust version accordingly)

$ configure

Configuration summary for nsca 291 01-27-2012

General Options

-------------------------

NSCA port 5667

NSCA user nagios

NSCA group nagios

$ make all

Solace Nagios Toolset Integration

19

63 Post install setup

1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca

This will generate required Config file and setup nsca as service

NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP

setup_nsca -s optpkgnsca-291

srcdir optpkgnsca-291 ok

target dir usrlocalnsca ok

Checking

hellip

Installing

hellip

Creating service

Installation complete

2 Start service

Now that the component is installed start it up

systemctl restart nsca

Solace Nagios Toolset Integration

20

64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl

ndashh for usage info

641 Sending Host Alert

send_ngalertpl -n vmr3 -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending host command vmr30 - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0

Output - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe

[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection

Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -

[Status=UP]

Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]

Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]

Solace Nagios Toolset Integration

21

642 Sending Service Alert

send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending service command vmr3hardware0 - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service

Description hardware Return Code 0 Output - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe

Nov 9 154116 centos7-vm nsca[9888] End of connection

Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND

PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]

Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]

Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]

Solace Nagios Toolset Integration

22

7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing

monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an

active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be

configured on both the server and the client end though the client end changes are relatively light

In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server

as client

71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that

network connectivity exists between the server and client hosts and the ports are open both ways

72 Installation

721 Server (CentOS)

In addition to Nagios server packages we would need to install the following These were already covered in previous

sections and listed here for completeness and for any standalone installs

NRPE

Nagios Pluggins

OpenSSL

yum install -y nrpe nagios-plugins-all openssl

722 Client (Ubuntu)

On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component

apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-

standard

Solace Nagios Toolset Integration

23

73 Post Install Configuration

731 Client Configuration

1 Add Nagios server IP to allowed_hosts in nrpecfg

grep ^allowed_hosts etcnagiosnrpecfg

allowed_hosts=127001192168562011921681223

2 Restart service

systemctl start nagios-nrpe-server

732 Server Configuration

1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in

ldquoSetup Scripts and Config Filesrdquo section

2 Update nagioscfg (etcnagios by default) and add the client config file

For eg

rootcentos7-vm confd tail etcnagiosnagioscfg

cfg_file=etcnagiosobjectsnrpe-test1cfg

3 Now verify the config and fix any errors

nagios -v etcnagiosnagioscfg

4 Restart services

systemctl restart nagios

systemctl restart nrpe

systemctl restart httpd

Solace Nagios Toolset Integration

24

74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot

below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status

75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and

differences

1 NCSA does passive check This mean client choses when to send the status to the server This allowes

fine grain control over the update timing and useful for near real time updates

2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically

This leads to simpler install

3 NRPE does active check The server pulls the info from the client periodically This is better suited for

centrally adminsterored policies

4 NRPE requires a daemon be runnion on all the clients to service NRPE requests

5 NCSA is not in active development anymore (no updates since 2013)

Solace Nagios Toolset Integration

25

8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD

(Round Robin Database) It can collect manage and display graphs of collected data

81 Prerequisites

811 Cacti Dependencies

Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use

MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration

unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not

required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files

82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are

instructions and materials available that can be readily used Here are some lnks

httpdocscactinetmanual0881_installation1_install_unix

httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-

fedora-17-12

httpwwwcactinetdownloadsdocshtmldebugginghtml

83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to

Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for

illustration which can be replaced with any monitoring using right scripts

Solace Nagios Toolset Integration

26

831 Add Custom Data Input method

Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the

solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script

need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected

form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot

do any validation on Output Field

8311 Data Input

8312 Input

8313 Output

Solace Nagios Toolset Integration

27

832 Add Custom Data Template

Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to

declare info for the specific queue (router name vpn name queue name and credentials)

Solace Nagios Toolset Integration

28

833 Add a custom Graph Template

Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph

Template Items)

Solace Nagios Toolset Integration

29

834 Add Solace Appliance as Device

Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to

run host specific services like ping test

8341 Add Data Source List

Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown

here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the

correspoinding RRD files for these resources

Solace Nagios Toolset Integration

30

8342 Add Graph List

Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the

specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool

Solace Nagios Toolset Integration

31

84 Verification

841 Basic Install

Once you have basic Cacti setup up and running you should be able to access the install and do further configuration

thru web admin page at

httpltyour-server-ipgtcacti

After configuration you would be able to see basic system metrics for the localhost that would look similar to this

Solace Nagios Toolset Integration

32

842 Solace host check

Once Solce router is added as a device from he Devices link you would be able view the router with basic health test

such as ping status

843 Solace VPN Metrics Sample

Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue

statistics showing queue depth for a single queue on a VPN

Solace Nagios Toolset Integration

33

As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of

aggregation of queue stats over various window period from hour to year

Solace Nagios Toolset Integration

34

9 Additional Topics

91 Admin Tasks

911 Creating custom Maps

Create custom maps for any outputs that need it for example turn the iface counter values into gauge

Add to usrlocalnagiosgraphetcnagiosgraphmap

Service type iface

outputIFACE OK - [Interface161EnabledyesLinkyes]

perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-

bytes=363571175913

perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)

and push s [ if_stats

[tx-pkts COUNTER int $1 ]

[rx-pkts COUNTER int $2 ]

[tx-bytes COUNTER int $3 ]

[rx-bytes COUNTER int $4 ] ]

912 Adding additional routers VPNs

Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run

mk_nagiossolacecfg to generate required solace plugin Config files

Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be

added into etcnagiosnagioscfg file

Note that the services need to be restarted for the changes to be effective

913 Putting Nagios on Verbose mode

Change debug level in Nagios config to 2048

grep ^debug_level etcnagiosnagioscfg

debug_level=2048

Now check the debug logs All commands executed and output will be logged over there

Solace Nagios Toolset Integration

35

tail varlognagiosnagiosdebug

[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING

[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w

usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --

queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet

[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w

usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --

vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet

[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING

914 Modifying iptables

When you Cant access http from outside or when telnet to server on port 80 fail

Check Iptables routes

iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT

iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT

iptables-save | tee etcsysconfigiptables

service iptables save

service iptables restart

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

16

53 Post Install Setup

54 Verification If the sample router info is successfully configured a host entry matching the appliance hostname from Config file will

be added to the web page

A sample entry is shown below

Solace Nagios Toolset Integration

17

Clicking on the graph icon to the right of the service name brings up the graphs with options A sample graph is shown

below

Solace Nagios Toolset Integration

18

6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the

external command file Nagios would periodically check the external command file and process them This ldquopassive

checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured

hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status

updates and alerting

61 Prerequisites

62 Installation

1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291

(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)

2 Untar and follow install instructions in the package

$ cd optpkg (or directory of your choice)

$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site

for latest package name and replace)

$ tar xvzf nsca-291targz (adjust version accordingly)

$ cd optpkgnsca-291 (adjust version accordingly)

$ configure

Configuration summary for nsca 291 01-27-2012

General Options

-------------------------

NSCA port 5667

NSCA user nagios

NSCA group nagios

$ make all

Solace Nagios Toolset Integration

19

63 Post install setup

1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca

This will generate required Config file and setup nsca as service

NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP

setup_nsca -s optpkgnsca-291

srcdir optpkgnsca-291 ok

target dir usrlocalnsca ok

Checking

hellip

Installing

hellip

Creating service

Installation complete

2 Start service

Now that the component is installed start it up

systemctl restart nsca

Solace Nagios Toolset Integration

20

64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl

ndashh for usage info

641 Sending Host Alert

send_ngalertpl -n vmr3 -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending host command vmr30 - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0

Output - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe

[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection

Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -

[Status=UP]

Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]

Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]

Solace Nagios Toolset Integration

21

642 Sending Service Alert

send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending service command vmr3hardware0 - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service

Description hardware Return Code 0 Output - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe

Nov 9 154116 centos7-vm nsca[9888] End of connection

Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND

PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]

Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]

Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]

Solace Nagios Toolset Integration

22

7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing

monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an

active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be

configured on both the server and the client end though the client end changes are relatively light

In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server

as client

71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that

network connectivity exists between the server and client hosts and the ports are open both ways

72 Installation

721 Server (CentOS)

In addition to Nagios server packages we would need to install the following These were already covered in previous

sections and listed here for completeness and for any standalone installs

NRPE

Nagios Pluggins

OpenSSL

yum install -y nrpe nagios-plugins-all openssl

722 Client (Ubuntu)

On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component

apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-

standard

Solace Nagios Toolset Integration

23

73 Post Install Configuration

731 Client Configuration

1 Add Nagios server IP to allowed_hosts in nrpecfg

grep ^allowed_hosts etcnagiosnrpecfg

allowed_hosts=127001192168562011921681223

2 Restart service

systemctl start nagios-nrpe-server

732 Server Configuration

1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in

ldquoSetup Scripts and Config Filesrdquo section

2 Update nagioscfg (etcnagios by default) and add the client config file

For eg

rootcentos7-vm confd tail etcnagiosnagioscfg

cfg_file=etcnagiosobjectsnrpe-test1cfg

3 Now verify the config and fix any errors

nagios -v etcnagiosnagioscfg

4 Restart services

systemctl restart nagios

systemctl restart nrpe

systemctl restart httpd

Solace Nagios Toolset Integration

24

74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot

below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status

75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and

differences

1 NCSA does passive check This mean client choses when to send the status to the server This allowes

fine grain control over the update timing and useful for near real time updates

2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically

This leads to simpler install

3 NRPE does active check The server pulls the info from the client periodically This is better suited for

centrally adminsterored policies

4 NRPE requires a daemon be runnion on all the clients to service NRPE requests

5 NCSA is not in active development anymore (no updates since 2013)

Solace Nagios Toolset Integration

25

8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD

(Round Robin Database) It can collect manage and display graphs of collected data

81 Prerequisites

811 Cacti Dependencies

Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use

MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration

unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not

required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files

82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are

instructions and materials available that can be readily used Here are some lnks

httpdocscactinetmanual0881_installation1_install_unix

httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-

fedora-17-12

httpwwwcactinetdownloadsdocshtmldebugginghtml

83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to

Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for

illustration which can be replaced with any monitoring using right scripts

Solace Nagios Toolset Integration

26

831 Add Custom Data Input method

Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the

solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script

need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected

form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot

do any validation on Output Field

8311 Data Input

8312 Input

8313 Output

Solace Nagios Toolset Integration

27

832 Add Custom Data Template

Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to

declare info for the specific queue (router name vpn name queue name and credentials)

Solace Nagios Toolset Integration

28

833 Add a custom Graph Template

Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph

Template Items)

Solace Nagios Toolset Integration

29

834 Add Solace Appliance as Device

Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to

run host specific services like ping test

8341 Add Data Source List

Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown

here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the

correspoinding RRD files for these resources

Solace Nagios Toolset Integration

30

8342 Add Graph List

Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the

specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool

Solace Nagios Toolset Integration

31

84 Verification

841 Basic Install

Once you have basic Cacti setup up and running you should be able to access the install and do further configuration

thru web admin page at

httpltyour-server-ipgtcacti

After configuration you would be able to see basic system metrics for the localhost that would look similar to this

Solace Nagios Toolset Integration

32

842 Solace host check

Once Solce router is added as a device from he Devices link you would be able view the router with basic health test

such as ping status

843 Solace VPN Metrics Sample

Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue

statistics showing queue depth for a single queue on a VPN

Solace Nagios Toolset Integration

33

As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of

aggregation of queue stats over various window period from hour to year

Solace Nagios Toolset Integration

34

9 Additional Topics

91 Admin Tasks

911 Creating custom Maps

Create custom maps for any outputs that need it for example turn the iface counter values into gauge

Add to usrlocalnagiosgraphetcnagiosgraphmap

Service type iface

outputIFACE OK - [Interface161EnabledyesLinkyes]

perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-

bytes=363571175913

perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)

and push s [ if_stats

[tx-pkts COUNTER int $1 ]

[rx-pkts COUNTER int $2 ]

[tx-bytes COUNTER int $3 ]

[rx-bytes COUNTER int $4 ] ]

912 Adding additional routers VPNs

Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run

mk_nagiossolacecfg to generate required solace plugin Config files

Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be

added into etcnagiosnagioscfg file

Note that the services need to be restarted for the changes to be effective

913 Putting Nagios on Verbose mode

Change debug level in Nagios config to 2048

grep ^debug_level etcnagiosnagioscfg

debug_level=2048

Now check the debug logs All commands executed and output will be logged over there

Solace Nagios Toolset Integration

35

tail varlognagiosnagiosdebug

[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING

[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w

usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --

queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet

[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w

usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --

vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet

[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING

914 Modifying iptables

When you Cant access http from outside or when telnet to server on port 80 fail

Check Iptables routes

iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT

iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT

iptables-save | tee etcsysconfigiptables

service iptables save

service iptables restart

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

17

Clicking on the graph icon to the right of the service name brings up the graphs with options A sample graph is shown

below

Solace Nagios Toolset Integration

18

6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the

external command file Nagios would periodically check the external command file and process them This ldquopassive

checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured

hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status

updates and alerting

61 Prerequisites

62 Installation

1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291

(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)

2 Untar and follow install instructions in the package

$ cd optpkg (or directory of your choice)

$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site

for latest package name and replace)

$ tar xvzf nsca-291targz (adjust version accordingly)

$ cd optpkgnsca-291 (adjust version accordingly)

$ configure

Configuration summary for nsca 291 01-27-2012

General Options

-------------------------

NSCA port 5667

NSCA user nagios

NSCA group nagios

$ make all

Solace Nagios Toolset Integration

19

63 Post install setup

1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca

This will generate required Config file and setup nsca as service

NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP

setup_nsca -s optpkgnsca-291

srcdir optpkgnsca-291 ok

target dir usrlocalnsca ok

Checking

hellip

Installing

hellip

Creating service

Installation complete

2 Start service

Now that the component is installed start it up

systemctl restart nsca

Solace Nagios Toolset Integration

20

64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl

ndashh for usage info

641 Sending Host Alert

send_ngalertpl -n vmr3 -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending host command vmr30 - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0

Output - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe

[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection

Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -

[Status=UP]

Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]

Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]

Solace Nagios Toolset Integration

21

642 Sending Service Alert

send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending service command vmr3hardware0 - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service

Description hardware Return Code 0 Output - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe

Nov 9 154116 centos7-vm nsca[9888] End of connection

Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND

PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]

Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]

Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]

Solace Nagios Toolset Integration

22

7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing

monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an

active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be

configured on both the server and the client end though the client end changes are relatively light

In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server

as client

71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that

network connectivity exists between the server and client hosts and the ports are open both ways

72 Installation

721 Server (CentOS)

In addition to Nagios server packages we would need to install the following These were already covered in previous

sections and listed here for completeness and for any standalone installs

NRPE

Nagios Pluggins

OpenSSL

yum install -y nrpe nagios-plugins-all openssl

722 Client (Ubuntu)

On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component

apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-

standard

Solace Nagios Toolset Integration

23

73 Post Install Configuration

731 Client Configuration

1 Add Nagios server IP to allowed_hosts in nrpecfg

grep ^allowed_hosts etcnagiosnrpecfg

allowed_hosts=127001192168562011921681223

2 Restart service

systemctl start nagios-nrpe-server

732 Server Configuration

1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in

ldquoSetup Scripts and Config Filesrdquo section

2 Update nagioscfg (etcnagios by default) and add the client config file

For eg

rootcentos7-vm confd tail etcnagiosnagioscfg

cfg_file=etcnagiosobjectsnrpe-test1cfg

3 Now verify the config and fix any errors

nagios -v etcnagiosnagioscfg

4 Restart services

systemctl restart nagios

systemctl restart nrpe

systemctl restart httpd

Solace Nagios Toolset Integration

24

74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot

below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status

75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and

differences

1 NCSA does passive check This mean client choses when to send the status to the server This allowes

fine grain control over the update timing and useful for near real time updates

2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically

This leads to simpler install

3 NRPE does active check The server pulls the info from the client periodically This is better suited for

centrally adminsterored policies

4 NRPE requires a daemon be runnion on all the clients to service NRPE requests

5 NCSA is not in active development anymore (no updates since 2013)

Solace Nagios Toolset Integration

25

8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD

(Round Robin Database) It can collect manage and display graphs of collected data

81 Prerequisites

811 Cacti Dependencies

Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use

MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration

unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not

required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files

82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are

instructions and materials available that can be readily used Here are some lnks

httpdocscactinetmanual0881_installation1_install_unix

httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-

fedora-17-12

httpwwwcactinetdownloadsdocshtmldebugginghtml

83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to

Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for

illustration which can be replaced with any monitoring using right scripts

Solace Nagios Toolset Integration

26

831 Add Custom Data Input method

Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the

solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script

need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected

form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot

do any validation on Output Field

8311 Data Input

8312 Input

8313 Output

Solace Nagios Toolset Integration

27

832 Add Custom Data Template

Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to

declare info for the specific queue (router name vpn name queue name and credentials)

Solace Nagios Toolset Integration

28

833 Add a custom Graph Template

Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph

Template Items)

Solace Nagios Toolset Integration

29

834 Add Solace Appliance as Device

Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to

run host specific services like ping test

8341 Add Data Source List

Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown

here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the

correspoinding RRD files for these resources

Solace Nagios Toolset Integration

30

8342 Add Graph List

Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the

specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool

Solace Nagios Toolset Integration

31

84 Verification

841 Basic Install

Once you have basic Cacti setup up and running you should be able to access the install and do further configuration

thru web admin page at

httpltyour-server-ipgtcacti

After configuration you would be able to see basic system metrics for the localhost that would look similar to this

Solace Nagios Toolset Integration

32

842 Solace host check

Once Solce router is added as a device from he Devices link you would be able view the router with basic health test

such as ping status

843 Solace VPN Metrics Sample

Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue

statistics showing queue depth for a single queue on a VPN

Solace Nagios Toolset Integration

33

As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of

aggregation of queue stats over various window period from hour to year

Solace Nagios Toolset Integration

34

9 Additional Topics

91 Admin Tasks

911 Creating custom Maps

Create custom maps for any outputs that need it for example turn the iface counter values into gauge

Add to usrlocalnagiosgraphetcnagiosgraphmap

Service type iface

outputIFACE OK - [Interface161EnabledyesLinkyes]

perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-

bytes=363571175913

perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)

and push s [ if_stats

[tx-pkts COUNTER int $1 ]

[rx-pkts COUNTER int $2 ]

[tx-bytes COUNTER int $3 ]

[rx-bytes COUNTER int $4 ] ]

912 Adding additional routers VPNs

Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run

mk_nagiossolacecfg to generate required solace plugin Config files

Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be

added into etcnagiosnagioscfg file

Note that the services need to be restarted for the changes to be effective

913 Putting Nagios on Verbose mode

Change debug level in Nagios config to 2048

grep ^debug_level etcnagiosnagioscfg

debug_level=2048

Now check the debug logs All commands executed and output will be logged over there

Solace Nagios Toolset Integration

35

tail varlognagiosnagiosdebug

[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING

[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w

usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --

queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet

[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w

usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --

vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet

[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING

914 Modifying iptables

When you Cant access http from outside or when telnet to server on port 80 fail

Check Iptables routes

iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT

iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT

iptables-save | tee etcsysconfigiptables

service iptables save

service iptables restart

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

18

6 NSCA Setup NSCA (Nagios Service check Acceptor) is a daemon that can accept the requests on behalf of Nagios and update the

external command file Nagios would periodically check the external command file and process them This ldquopassive

checkrdquo is a way to push the external status and alerts into Nagios instead of Nagios pulling the status for configured

hosts and services ldquoactivelyrdquo The passive check offers more fine grained intervals and useful for near real time status

updates and alerting

61 Prerequisites

62 Installation

1 Download NSCA plugin from the Nagios Exchange The current version as of this writing is 291

(httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz)

2 Untar and follow install instructions in the package

$ cd optpkg (or directory of your choice)

$ wget httpprdownloadssourceforgenetsourceforgenagiosnsca-291targz (check the nsca site

for latest package name and replace)

$ tar xvzf nsca-291targz (adjust version accordingly)

$ cd optpkgnsca-291 (adjust version accordingly)

$ configure

Configuration summary for nsca 291 01-27-2012

General Options

-------------------------

NSCA port 5667

NSCA user nagios

NSCA group nagios

$ make all

Solace Nagios Toolset Integration

19

63 Post install setup

1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca

This will generate required Config file and setup nsca as service

NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP

setup_nsca -s optpkgnsca-291

srcdir optpkgnsca-291 ok

target dir usrlocalnsca ok

Checking

hellip

Installing

hellip

Creating service

Installation complete

2 Start service

Now that the component is installed start it up

systemctl restart nsca

Solace Nagios Toolset Integration

20

64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl

ndashh for usage info

641 Sending Host Alert

send_ngalertpl -n vmr3 -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending host command vmr30 - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0

Output - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe

[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection

Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -

[Status=UP]

Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]

Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]

Solace Nagios Toolset Integration

21

642 Sending Service Alert

send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending service command vmr3hardware0 - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service

Description hardware Return Code 0 Output - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe

Nov 9 154116 centos7-vm nsca[9888] End of connection

Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND

PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]

Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]

Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]

Solace Nagios Toolset Integration

22

7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing

monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an

active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be

configured on both the server and the client end though the client end changes are relatively light

In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server

as client

71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that

network connectivity exists between the server and client hosts and the ports are open both ways

72 Installation

721 Server (CentOS)

In addition to Nagios server packages we would need to install the following These were already covered in previous

sections and listed here for completeness and for any standalone installs

NRPE

Nagios Pluggins

OpenSSL

yum install -y nrpe nagios-plugins-all openssl

722 Client (Ubuntu)

On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component

apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-

standard

Solace Nagios Toolset Integration

23

73 Post Install Configuration

731 Client Configuration

1 Add Nagios server IP to allowed_hosts in nrpecfg

grep ^allowed_hosts etcnagiosnrpecfg

allowed_hosts=127001192168562011921681223

2 Restart service

systemctl start nagios-nrpe-server

732 Server Configuration

1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in

ldquoSetup Scripts and Config Filesrdquo section

2 Update nagioscfg (etcnagios by default) and add the client config file

For eg

rootcentos7-vm confd tail etcnagiosnagioscfg

cfg_file=etcnagiosobjectsnrpe-test1cfg

3 Now verify the config and fix any errors

nagios -v etcnagiosnagioscfg

4 Restart services

systemctl restart nagios

systemctl restart nrpe

systemctl restart httpd

Solace Nagios Toolset Integration

24

74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot

below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status

75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and

differences

1 NCSA does passive check This mean client choses when to send the status to the server This allowes

fine grain control over the update timing and useful for near real time updates

2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically

This leads to simpler install

3 NRPE does active check The server pulls the info from the client periodically This is better suited for

centrally adminsterored policies

4 NRPE requires a daemon be runnion on all the clients to service NRPE requests

5 NCSA is not in active development anymore (no updates since 2013)

Solace Nagios Toolset Integration

25

8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD

(Round Robin Database) It can collect manage and display graphs of collected data

81 Prerequisites

811 Cacti Dependencies

Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use

MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration

unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not

required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files

82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are

instructions and materials available that can be readily used Here are some lnks

httpdocscactinetmanual0881_installation1_install_unix

httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-

fedora-17-12

httpwwwcactinetdownloadsdocshtmldebugginghtml

83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to

Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for

illustration which can be replaced with any monitoring using right scripts

Solace Nagios Toolset Integration

26

831 Add Custom Data Input method

Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the

solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script

need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected

form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot

do any validation on Output Field

8311 Data Input

8312 Input

8313 Output

Solace Nagios Toolset Integration

27

832 Add Custom Data Template

Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to

declare info for the specific queue (router name vpn name queue name and credentials)

Solace Nagios Toolset Integration

28

833 Add a custom Graph Template

Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph

Template Items)

Solace Nagios Toolset Integration

29

834 Add Solace Appliance as Device

Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to

run host specific services like ping test

8341 Add Data Source List

Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown

here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the

correspoinding RRD files for these resources

Solace Nagios Toolset Integration

30

8342 Add Graph List

Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the

specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool

Solace Nagios Toolset Integration

31

84 Verification

841 Basic Install

Once you have basic Cacti setup up and running you should be able to access the install and do further configuration

thru web admin page at

httpltyour-server-ipgtcacti

After configuration you would be able to see basic system metrics for the localhost that would look similar to this

Solace Nagios Toolset Integration

32

842 Solace host check

Once Solce router is added as a device from he Devices link you would be able view the router with basic health test

such as ping status

843 Solace VPN Metrics Sample

Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue

statistics showing queue depth for a single queue on a VPN

Solace Nagios Toolset Integration

33

As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of

aggregation of queue stats over various window period from hour to year

Solace Nagios Toolset Integration

34

9 Additional Topics

91 Admin Tasks

911 Creating custom Maps

Create custom maps for any outputs that need it for example turn the iface counter values into gauge

Add to usrlocalnagiosgraphetcnagiosgraphmap

Service type iface

outputIFACE OK - [Interface161EnabledyesLinkyes]

perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-

bytes=363571175913

perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)

and push s [ if_stats

[tx-pkts COUNTER int $1 ]

[rx-pkts COUNTER int $2 ]

[tx-bytes COUNTER int $3 ]

[rx-bytes COUNTER int $4 ] ]

912 Adding additional routers VPNs

Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run

mk_nagiossolacecfg to generate required solace plugin Config files

Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be

added into etcnagiosnagioscfg file

Note that the services need to be restarted for the changes to be effective

913 Putting Nagios on Verbose mode

Change debug level in Nagios config to 2048

grep ^debug_level etcnagiosnagioscfg

debug_level=2048

Now check the debug logs All commands executed and output will be logged over there

Solace Nagios Toolset Integration

35

tail varlognagiosnagiosdebug

[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING

[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w

usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --

queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet

[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w

usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --

vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet

[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING

914 Modifying iptables

When you Cant access http from outside or when telnet to server on port 80 fail

Check Iptables routes

iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT

iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT

iptables-save | tee etcsysconfigiptables

service iptables save

service iptables restart

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

19

63 Post install setup

1 Run setup_nsca script with source and target directories The default target dir is usrlocalnsca

This will generate required Config file and setup nsca as service

NOTE Check Config file usrlocalnscacfgnscacfg and make changes if required esp to host IP

setup_nsca -s optpkgnsca-291

srcdir optpkgnsca-291 ok

target dir usrlocalnsca ok

Checking

hellip

Installing

hellip

Creating service

Installation complete

2 Start service

Now that the component is installed start it up

systemctl restart nsca

Solace Nagios Toolset Integration

20

64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl

ndashh for usage info

641 Sending Host Alert

send_ngalertpl -n vmr3 -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending host command vmr30 - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0

Output - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe

[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection

Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -

[Status=UP]

Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]

Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]

Solace Nagios Toolset Integration

21

642 Sending Service Alert

send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending service command vmr3hardware0 - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service

Description hardware Return Code 0 Output - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe

Nov 9 154116 centos7-vm nsca[9888] End of connection

Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND

PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]

Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]

Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]

Solace Nagios Toolset Integration

22

7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing

monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an

active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be

configured on both the server and the client end though the client end changes are relatively light

In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server

as client

71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that

network connectivity exists between the server and client hosts and the ports are open both ways

72 Installation

721 Server (CentOS)

In addition to Nagios server packages we would need to install the following These were already covered in previous

sections and listed here for completeness and for any standalone installs

NRPE

Nagios Pluggins

OpenSSL

yum install -y nrpe nagios-plugins-all openssl

722 Client (Ubuntu)

On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component

apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-

standard

Solace Nagios Toolset Integration

23

73 Post Install Configuration

731 Client Configuration

1 Add Nagios server IP to allowed_hosts in nrpecfg

grep ^allowed_hosts etcnagiosnrpecfg

allowed_hosts=127001192168562011921681223

2 Restart service

systemctl start nagios-nrpe-server

732 Server Configuration

1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in

ldquoSetup Scripts and Config Filesrdquo section

2 Update nagioscfg (etcnagios by default) and add the client config file

For eg

rootcentos7-vm confd tail etcnagiosnagioscfg

cfg_file=etcnagiosobjectsnrpe-test1cfg

3 Now verify the config and fix any errors

nagios -v etcnagiosnagioscfg

4 Restart services

systemctl restart nagios

systemctl restart nrpe

systemctl restart httpd

Solace Nagios Toolset Integration

24

74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot

below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status

75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and

differences

1 NCSA does passive check This mean client choses when to send the status to the server This allowes

fine grain control over the update timing and useful for near real time updates

2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically

This leads to simpler install

3 NRPE does active check The server pulls the info from the client periodically This is better suited for

centrally adminsterored policies

4 NRPE requires a daemon be runnion on all the clients to service NRPE requests

5 NCSA is not in active development anymore (no updates since 2013)

Solace Nagios Toolset Integration

25

8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD

(Round Robin Database) It can collect manage and display graphs of collected data

81 Prerequisites

811 Cacti Dependencies

Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use

MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration

unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not

required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files

82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are

instructions and materials available that can be readily used Here are some lnks

httpdocscactinetmanual0881_installation1_install_unix

httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-

fedora-17-12

httpwwwcactinetdownloadsdocshtmldebugginghtml

83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to

Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for

illustration which can be replaced with any monitoring using right scripts

Solace Nagios Toolset Integration

26

831 Add Custom Data Input method

Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the

solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script

need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected

form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot

do any validation on Output Field

8311 Data Input

8312 Input

8313 Output

Solace Nagios Toolset Integration

27

832 Add Custom Data Template

Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to

declare info for the specific queue (router name vpn name queue name and credentials)

Solace Nagios Toolset Integration

28

833 Add a custom Graph Template

Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph

Template Items)

Solace Nagios Toolset Integration

29

834 Add Solace Appliance as Device

Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to

run host specific services like ping test

8341 Add Data Source List

Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown

here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the

correspoinding RRD files for these resources

Solace Nagios Toolset Integration

30

8342 Add Graph List

Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the

specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool

Solace Nagios Toolset Integration

31

84 Verification

841 Basic Install

Once you have basic Cacti setup up and running you should be able to access the install and do further configuration

thru web admin page at

httpltyour-server-ipgtcacti

After configuration you would be able to see basic system metrics for the localhost that would look similar to this

Solace Nagios Toolset Integration

32

842 Solace host check

Once Solce router is added as a device from he Devices link you would be able view the router with basic health test

such as ping status

843 Solace VPN Metrics Sample

Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue

statistics showing queue depth for a single queue on a VPN

Solace Nagios Toolset Integration

33

As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of

aggregation of queue stats over various window period from hour to year

Solace Nagios Toolset Integration

34

9 Additional Topics

91 Admin Tasks

911 Creating custom Maps

Create custom maps for any outputs that need it for example turn the iface counter values into gauge

Add to usrlocalnagiosgraphetcnagiosgraphmap

Service type iface

outputIFACE OK - [Interface161EnabledyesLinkyes]

perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-

bytes=363571175913

perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)

and push s [ if_stats

[tx-pkts COUNTER int $1 ]

[rx-pkts COUNTER int $2 ]

[tx-bytes COUNTER int $3 ]

[rx-bytes COUNTER int $4 ] ]

912 Adding additional routers VPNs

Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run

mk_nagiossolacecfg to generate required solace plugin Config files

Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be

added into etcnagiosnagioscfg file

Note that the services need to be restarted for the changes to be effective

913 Putting Nagios on Verbose mode

Change debug level in Nagios config to 2048

grep ^debug_level etcnagiosnagioscfg

debug_level=2048

Now check the debug logs All commands executed and output will be logged over there

Solace Nagios Toolset Integration

35

tail varlognagiosnagiosdebug

[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING

[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w

usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --

queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet

[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w

usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --

vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet

[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING

914 Modifying iptables

When you Cant access http from outside or when telnet to server on port 80 fail

Check Iptables routes

iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT

iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT

iptables-save | tee etcsysconfigiptables

service iptables save

service iptables restart

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

20

64 Verification Use send_alert or send_ngalertpl wrapper to send an alert to remote host and test Use send_alert ndashh send_ngalertpl

ndashh for usage info

641 Sending Host Alert

send_ngalertpl -n vmr3 -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending host command vmr30 - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154030 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154030 centos7-vm nsca[9888] HOST CHECK -gt Host Name vmr3 Return Code 0

Output - [Status=UP]

Nov 9 154030 centos7-vm nsca[9888] Attempting to write to nagios command pipe

[rootcentos7-vm bin] Nov 9 154030 centos7-vm nsca[9888] End of connection

Nov 9 154030 centos7-vm nagios EXTERNAL COMMAND PROCESS_HOST_CHECK_RESULTvmr30 -

[Status=UP]

Nov 9 154033 centos7-vm nagios PASSIVE HOST CHECK vmr30- [Status=UP]

Nov 9 154033 centos7-vm nagios HOST ALERT vmr3UPHARD1- [Status=UP]

Solace Nagios Toolset Integration

21

642 Sending Service Alert

send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending service command vmr3hardware0 - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service

Description hardware Return Code 0 Output - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe

Nov 9 154116 centos7-vm nsca[9888] End of connection

Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND

PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]

Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]

Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]

Solace Nagios Toolset Integration

22

7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing

monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an

active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be

configured on both the server and the client end though the client end changes are relatively light

In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server

as client

71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that

network connectivity exists between the server and client hosts and the ports are open both ways

72 Installation

721 Server (CentOS)

In addition to Nagios server packages we would need to install the following These were already covered in previous

sections and listed here for completeness and for any standalone installs

NRPE

Nagios Pluggins

OpenSSL

yum install -y nrpe nagios-plugins-all openssl

722 Client (Ubuntu)

On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component

apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-

standard

Solace Nagios Toolset Integration

23

73 Post Install Configuration

731 Client Configuration

1 Add Nagios server IP to allowed_hosts in nrpecfg

grep ^allowed_hosts etcnagiosnrpecfg

allowed_hosts=127001192168562011921681223

2 Restart service

systemctl start nagios-nrpe-server

732 Server Configuration

1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in

ldquoSetup Scripts and Config Filesrdquo section

2 Update nagioscfg (etcnagios by default) and add the client config file

For eg

rootcentos7-vm confd tail etcnagiosnagioscfg

cfg_file=etcnagiosobjectsnrpe-test1cfg

3 Now verify the config and fix any errors

nagios -v etcnagiosnagioscfg

4 Restart services

systemctl restart nagios

systemctl restart nrpe

systemctl restart httpd

Solace Nagios Toolset Integration

24

74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot

below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status

75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and

differences

1 NCSA does passive check This mean client choses when to send the status to the server This allowes

fine grain control over the update timing and useful for near real time updates

2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically

This leads to simpler install

3 NRPE does active check The server pulls the info from the client periodically This is better suited for

centrally adminsterored policies

4 NRPE requires a daemon be runnion on all the clients to service NRPE requests

5 NCSA is not in active development anymore (no updates since 2013)

Solace Nagios Toolset Integration

25

8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD

(Round Robin Database) It can collect manage and display graphs of collected data

81 Prerequisites

811 Cacti Dependencies

Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use

MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration

unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not

required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files

82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are

instructions and materials available that can be readily used Here are some lnks

httpdocscactinetmanual0881_installation1_install_unix

httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-

fedora-17-12

httpwwwcactinetdownloadsdocshtmldebugginghtml

83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to

Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for

illustration which can be replaced with any monitoring using right scripts

Solace Nagios Toolset Integration

26

831 Add Custom Data Input method

Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the

solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script

need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected

form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot

do any validation on Output Field

8311 Data Input

8312 Input

8313 Output

Solace Nagios Toolset Integration

27

832 Add Custom Data Template

Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to

declare info for the specific queue (router name vpn name queue name and credentials)

Solace Nagios Toolset Integration

28

833 Add a custom Graph Template

Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph

Template Items)

Solace Nagios Toolset Integration

29

834 Add Solace Appliance as Device

Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to

run host specific services like ping test

8341 Add Data Source List

Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown

here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the

correspoinding RRD files for these resources

Solace Nagios Toolset Integration

30

8342 Add Graph List

Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the

specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool

Solace Nagios Toolset Integration

31

84 Verification

841 Basic Install

Once you have basic Cacti setup up and running you should be able to access the install and do further configuration

thru web admin page at

httpltyour-server-ipgtcacti

After configuration you would be able to see basic system metrics for the localhost that would look similar to this

Solace Nagios Toolset Integration

32

842 Solace host check

Once Solce router is added as a device from he Devices link you would be able view the router with basic health test

such as ping status

843 Solace VPN Metrics Sample

Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue

statistics showing queue depth for a single queue on a VPN

Solace Nagios Toolset Integration

33

As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of

aggregation of queue stats over various window period from hour to year

Solace Nagios Toolset Integration

34

9 Additional Topics

91 Admin Tasks

911 Creating custom Maps

Create custom maps for any outputs that need it for example turn the iface counter values into gauge

Add to usrlocalnagiosgraphetcnagiosgraphmap

Service type iface

outputIFACE OK - [Interface161EnabledyesLinkyes]

perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-

bytes=363571175913

perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)

and push s [ if_stats

[tx-pkts COUNTER int $1 ]

[rx-pkts COUNTER int $2 ]

[tx-bytes COUNTER int $3 ]

[rx-bytes COUNTER int $4 ] ]

912 Adding additional routers VPNs

Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run

mk_nagiossolacecfg to generate required solace plugin Config files

Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be

added into etcnagiosnagioscfg file

Note that the services need to be restarted for the changes to be effective

913 Putting Nagios on Verbose mode

Change debug level in Nagios config to 2048

grep ^debug_level etcnagiosnagioscfg

debug_level=2048

Now check the debug logs All commands executed and output will be logged over there

Solace Nagios Toolset Integration

35

tail varlognagiosnagiosdebug

[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING

[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w

usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --

queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet

[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w

usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --

vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet

[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING

914 Modifying iptables

When you Cant access http from outside or when telnet to server on port 80 fail

Check Iptables routes

iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT

iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT

iptables-save | tee etcsysconfigiptables

service iptables save

service iptables restart

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

21

642 Sending Service Alert

send_ngalertpl -n vmr3 -s hardware -c OK -i Status=UP

Reading server config usrlocalnscacfgnsca_servercfg

starting send_nsca client

sending service command vmr3hardware0 - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Handling the connection

1 data packet(s) sent to host successfully

varlogmessages Logs

Nov 9 154116 centos7-vm nsca[9888] Time difference in packet 0 seconds for host vmr3

Nov 9 154116 centos7-vm nsca[9888] SERVICE CHECK -gt Host Name vmr3 Service

Description hardware Return Code 0 Output - [Status=UP]

Nov 9 154116 centos7-vm nsca[9888] Attempting to write to nagios command pipe

Nov 9 154116 centos7-vm nsca[9888] End of connection

Nov 9 154116 centos7-vm nagios EXTERNAL COMMAND

PROCESS_SERVICE_CHECK_RESULTvmr3hardware0 - [Status=UP]

Nov 9 154123 centos7-vm nagios PASSIVE SERVICE CHECK vmr3hardware0- [Status=UP]

Nov 9 154123 centos7-vm nagios SERVICE ALERT vmr3hardwareOKSOFT2- [Status=UP]

Solace Nagios Toolset Integration

22

7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing

monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an

active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be

configured on both the server and the client end though the client end changes are relatively light

In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server

as client

71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that

network connectivity exists between the server and client hosts and the ports are open both ways

72 Installation

721 Server (CentOS)

In addition to Nagios server packages we would need to install the following These were already covered in previous

sections and listed here for completeness and for any standalone installs

NRPE

Nagios Pluggins

OpenSSL

yum install -y nrpe nagios-plugins-all openssl

722 Client (Ubuntu)

On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component

apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-

standard

Solace Nagios Toolset Integration

23

73 Post Install Configuration

731 Client Configuration

1 Add Nagios server IP to allowed_hosts in nrpecfg

grep ^allowed_hosts etcnagiosnrpecfg

allowed_hosts=127001192168562011921681223

2 Restart service

systemctl start nagios-nrpe-server

732 Server Configuration

1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in

ldquoSetup Scripts and Config Filesrdquo section

2 Update nagioscfg (etcnagios by default) and add the client config file

For eg

rootcentos7-vm confd tail etcnagiosnagioscfg

cfg_file=etcnagiosobjectsnrpe-test1cfg

3 Now verify the config and fix any errors

nagios -v etcnagiosnagioscfg

4 Restart services

systemctl restart nagios

systemctl restart nrpe

systemctl restart httpd

Solace Nagios Toolset Integration

24

74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot

below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status

75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and

differences

1 NCSA does passive check This mean client choses when to send the status to the server This allowes

fine grain control over the update timing and useful for near real time updates

2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically

This leads to simpler install

3 NRPE does active check The server pulls the info from the client periodically This is better suited for

centrally adminsterored policies

4 NRPE requires a daemon be runnion on all the clients to service NRPE requests

5 NCSA is not in active development anymore (no updates since 2013)

Solace Nagios Toolset Integration

25

8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD

(Round Robin Database) It can collect manage and display graphs of collected data

81 Prerequisites

811 Cacti Dependencies

Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use

MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration

unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not

required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files

82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are

instructions and materials available that can be readily used Here are some lnks

httpdocscactinetmanual0881_installation1_install_unix

httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-

fedora-17-12

httpwwwcactinetdownloadsdocshtmldebugginghtml

83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to

Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for

illustration which can be replaced with any monitoring using right scripts

Solace Nagios Toolset Integration

26

831 Add Custom Data Input method

Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the

solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script

need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected

form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot

do any validation on Output Field

8311 Data Input

8312 Input

8313 Output

Solace Nagios Toolset Integration

27

832 Add Custom Data Template

Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to

declare info for the specific queue (router name vpn name queue name and credentials)

Solace Nagios Toolset Integration

28

833 Add a custom Graph Template

Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph

Template Items)

Solace Nagios Toolset Integration

29

834 Add Solace Appliance as Device

Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to

run host specific services like ping test

8341 Add Data Source List

Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown

here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the

correspoinding RRD files for these resources

Solace Nagios Toolset Integration

30

8342 Add Graph List

Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the

specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool

Solace Nagios Toolset Integration

31

84 Verification

841 Basic Install

Once you have basic Cacti setup up and running you should be able to access the install and do further configuration

thru web admin page at

httpltyour-server-ipgtcacti

After configuration you would be able to see basic system metrics for the localhost that would look similar to this

Solace Nagios Toolset Integration

32

842 Solace host check

Once Solce router is added as a device from he Devices link you would be able view the router with basic health test

such as ping status

843 Solace VPN Metrics Sample

Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue

statistics showing queue depth for a single queue on a VPN

Solace Nagios Toolset Integration

33

As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of

aggregation of queue stats over various window period from hour to year

Solace Nagios Toolset Integration

34

9 Additional Topics

91 Admin Tasks

911 Creating custom Maps

Create custom maps for any outputs that need it for example turn the iface counter values into gauge

Add to usrlocalnagiosgraphetcnagiosgraphmap

Service type iface

outputIFACE OK - [Interface161EnabledyesLinkyes]

perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-

bytes=363571175913

perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)

and push s [ if_stats

[tx-pkts COUNTER int $1 ]

[rx-pkts COUNTER int $2 ]

[tx-bytes COUNTER int $3 ]

[rx-bytes COUNTER int $4 ] ]

912 Adding additional routers VPNs

Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run

mk_nagiossolacecfg to generate required solace plugin Config files

Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be

added into etcnagiosnagioscfg file

Note that the services need to be restarted for the changes to be effective

913 Putting Nagios on Verbose mode

Change debug level in Nagios config to 2048

grep ^debug_level etcnagiosnagioscfg

debug_level=2048

Now check the debug logs All commands executed and output will be logged over there

Solace Nagios Toolset Integration

35

tail varlognagiosnagiosdebug

[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING

[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w

usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --

queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet

[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w

usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --

vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet

[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING

914 Modifying iptables

When you Cant access http from outside or when telnet to server on port 80 fail

Check Iptables routes

iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT

iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT

iptables-save | tee etcsysconfigiptables

service iptables save

service iptables restart

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

22

7 NRPE Setup NRPE (Nagios Remote Plugin Executor) enables executing remote plugins on other LinuxUnix machines allowing

monitoring remote resources such as disk usage CPU load etc Unlike NCSA which is a passive check this is an

active check where the Nagios server pulls the information from remote hosts periodically This requires that NRPE be

configured on both the server and the client end though the client end changes are relatively light

In the following example we will continue to use the CenOS 7 server as Nagios server and conigure an Ubuntu server

as client

71 Prerequisites NRPE requires that the client and server communicate over specified port Default NRPE port is 5666 Ensure that

network connectivity exists between the server and client hosts and the ports are open both ways

72 Installation

721 Server (CentOS)

In addition to Nagios server packages we would need to install the following These were already covered in previous

sections and listed here for completeness and for any standalone installs

NRPE

Nagios Pluggins

OpenSSL

yum install -y nrpe nagios-plugins-all openssl

722 Client (Ubuntu)

On the remote client hosts we would need to install the Nagios plugins and NRPE daemon component

apt-get install openssl nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-

standard

Solace Nagios Toolset Integration

23

73 Post Install Configuration

731 Client Configuration

1 Add Nagios server IP to allowed_hosts in nrpecfg

grep ^allowed_hosts etcnagiosnrpecfg

allowed_hosts=127001192168562011921681223

2 Restart service

systemctl start nagios-nrpe-server

732 Server Configuration

1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in

ldquoSetup Scripts and Config Filesrdquo section

2 Update nagioscfg (etcnagios by default) and add the client config file

For eg

rootcentos7-vm confd tail etcnagiosnagioscfg

cfg_file=etcnagiosobjectsnrpe-test1cfg

3 Now verify the config and fix any errors

nagios -v etcnagiosnagioscfg

4 Restart services

systemctl restart nagios

systemctl restart nrpe

systemctl restart httpd

Solace Nagios Toolset Integration

24

74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot

below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status

75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and

differences

1 NCSA does passive check This mean client choses when to send the status to the server This allowes

fine grain control over the update timing and useful for near real time updates

2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically

This leads to simpler install

3 NRPE does active check The server pulls the info from the client periodically This is better suited for

centrally adminsterored policies

4 NRPE requires a daemon be runnion on all the clients to service NRPE requests

5 NCSA is not in active development anymore (no updates since 2013)

Solace Nagios Toolset Integration

25

8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD

(Round Robin Database) It can collect manage and display graphs of collected data

81 Prerequisites

811 Cacti Dependencies

Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use

MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration

unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not

required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files

82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are

instructions and materials available that can be readily used Here are some lnks

httpdocscactinetmanual0881_installation1_install_unix

httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-

fedora-17-12

httpwwwcactinetdownloadsdocshtmldebugginghtml

83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to

Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for

illustration which can be replaced with any monitoring using right scripts

Solace Nagios Toolset Integration

26

831 Add Custom Data Input method

Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the

solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script

need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected

form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot

do any validation on Output Field

8311 Data Input

8312 Input

8313 Output

Solace Nagios Toolset Integration

27

832 Add Custom Data Template

Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to

declare info for the specific queue (router name vpn name queue name and credentials)

Solace Nagios Toolset Integration

28

833 Add a custom Graph Template

Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph

Template Items)

Solace Nagios Toolset Integration

29

834 Add Solace Appliance as Device

Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to

run host specific services like ping test

8341 Add Data Source List

Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown

here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the

correspoinding RRD files for these resources

Solace Nagios Toolset Integration

30

8342 Add Graph List

Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the

specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool

Solace Nagios Toolset Integration

31

84 Verification

841 Basic Install

Once you have basic Cacti setup up and running you should be able to access the install and do further configuration

thru web admin page at

httpltyour-server-ipgtcacti

After configuration you would be able to see basic system metrics for the localhost that would look similar to this

Solace Nagios Toolset Integration

32

842 Solace host check

Once Solce router is added as a device from he Devices link you would be able view the router with basic health test

such as ping status

843 Solace VPN Metrics Sample

Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue

statistics showing queue depth for a single queue on a VPN

Solace Nagios Toolset Integration

33

As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of

aggregation of queue stats over various window period from hour to year

Solace Nagios Toolset Integration

34

9 Additional Topics

91 Admin Tasks

911 Creating custom Maps

Create custom maps for any outputs that need it for example turn the iface counter values into gauge

Add to usrlocalnagiosgraphetcnagiosgraphmap

Service type iface

outputIFACE OK - [Interface161EnabledyesLinkyes]

perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-

bytes=363571175913

perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)

and push s [ if_stats

[tx-pkts COUNTER int $1 ]

[rx-pkts COUNTER int $2 ]

[tx-bytes COUNTER int $3 ]

[rx-bytes COUNTER int $4 ] ]

912 Adding additional routers VPNs

Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run

mk_nagiossolacecfg to generate required solace plugin Config files

Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be

added into etcnagiosnagioscfg file

Note that the services need to be restarted for the changes to be effective

913 Putting Nagios on Verbose mode

Change debug level in Nagios config to 2048

grep ^debug_level etcnagiosnagioscfg

debug_level=2048

Now check the debug logs All commands executed and output will be logged over there

Solace Nagios Toolset Integration

35

tail varlognagiosnagiosdebug

[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING

[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w

usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --

queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet

[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w

usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --

vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet

[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING

914 Modifying iptables

When you Cant access http from outside or when telnet to server on port 80 fail

Check Iptables routes

iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT

iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT

iptables-save | tee etcsysconfigiptables

service iptables save

service iptables restart

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

23

73 Post Install Configuration

731 Client Configuration

1 Add Nagios server IP to allowed_hosts in nrpecfg

grep ^allowed_hosts etcnagiosnrpecfg

allowed_hosts=127001192168562011921681223

2 Restart service

systemctl start nagios-nrpe-server

732 Server Configuration

1 Add config file per client under nagios objects directory A sample config file for ubuntu server is given in

ldquoSetup Scripts and Config Filesrdquo section

2 Update nagioscfg (etcnagios by default) and add the client config file

For eg

rootcentos7-vm confd tail etcnagiosnagioscfg

cfg_file=etcnagiosobjectsnrpe-test1cfg

3 Now verify the config and fix any errors

nagios -v etcnagiosnagioscfg

4 Restart services

systemctl restart nagios

systemctl restart nrpe

systemctl restart httpd

Solace Nagios Toolset Integration

24

74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot

below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status

75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and

differences

1 NCSA does passive check This mean client choses when to send the status to the server This allowes

fine grain control over the update timing and useful for near real time updates

2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically

This leads to simpler install

3 NRPE does active check The server pulls the info from the client periodically This is better suited for

centrally adminsterored policies

4 NRPE requires a daemon be runnion on all the clients to service NRPE requests

5 NCSA is not in active development anymore (no updates since 2013)

Solace Nagios Toolset Integration

25

8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD

(Round Robin Database) It can collect manage and display graphs of collected data

81 Prerequisites

811 Cacti Dependencies

Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use

MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration

unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not

required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files

82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are

instructions and materials available that can be readily used Here are some lnks

httpdocscactinetmanual0881_installation1_install_unix

httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-

fedora-17-12

httpwwwcactinetdownloadsdocshtmldebugginghtml

83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to

Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for

illustration which can be replaced with any monitoring using right scripts

Solace Nagios Toolset Integration

26

831 Add Custom Data Input method

Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the

solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script

need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected

form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot

do any validation on Output Field

8311 Data Input

8312 Input

8313 Output

Solace Nagios Toolset Integration

27

832 Add Custom Data Template

Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to

declare info for the specific queue (router name vpn name queue name and credentials)

Solace Nagios Toolset Integration

28

833 Add a custom Graph Template

Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph

Template Items)

Solace Nagios Toolset Integration

29

834 Add Solace Appliance as Device

Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to

run host specific services like ping test

8341 Add Data Source List

Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown

here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the

correspoinding RRD files for these resources

Solace Nagios Toolset Integration

30

8342 Add Graph List

Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the

specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool

Solace Nagios Toolset Integration

31

84 Verification

841 Basic Install

Once you have basic Cacti setup up and running you should be able to access the install and do further configuration

thru web admin page at

httpltyour-server-ipgtcacti

After configuration you would be able to see basic system metrics for the localhost that would look similar to this

Solace Nagios Toolset Integration

32

842 Solace host check

Once Solce router is added as a device from he Devices link you would be able view the router with basic health test

such as ping status

843 Solace VPN Metrics Sample

Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue

statistics showing queue depth for a single queue on a VPN

Solace Nagios Toolset Integration

33

As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of

aggregation of queue stats over various window period from hour to year

Solace Nagios Toolset Integration

34

9 Additional Topics

91 Admin Tasks

911 Creating custom Maps

Create custom maps for any outputs that need it for example turn the iface counter values into gauge

Add to usrlocalnagiosgraphetcnagiosgraphmap

Service type iface

outputIFACE OK - [Interface161EnabledyesLinkyes]

perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-

bytes=363571175913

perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)

and push s [ if_stats

[tx-pkts COUNTER int $1 ]

[rx-pkts COUNTER int $2 ]

[tx-bytes COUNTER int $3 ]

[rx-bytes COUNTER int $4 ] ]

912 Adding additional routers VPNs

Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run

mk_nagiossolacecfg to generate required solace plugin Config files

Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be

added into etcnagiosnagioscfg file

Note that the services need to be restarted for the changes to be effective

913 Putting Nagios on Verbose mode

Change debug level in Nagios config to 2048

grep ^debug_level etcnagiosnagioscfg

debug_level=2048

Now check the debug logs All commands executed and output will be logged over there

Solace Nagios Toolset Integration

35

tail varlognagiosnagiosdebug

[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING

[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w

usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --

queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet

[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w

usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --

vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet

[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING

914 Modifying iptables

When you Cant access http from outside or when telnet to server on port 80 fail

Check Iptables routes

iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT

iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT

iptables-save | tee etcsysconfigiptables

service iptables save

service iptables restart

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

24

74 Verification After susseful NRPE configuration the newly added host(s) would appear on the Nagios dashboard In the screenshot

below the added NRPE client (ubuntu-svr) is shown with ping and other NRPE service status

75 NCSA vs NRPE Both NCSA and NRPE enable remote host monitoring with Nagios This section looks at the advantages and

differences

1 NCSA does passive check This mean client choses when to send the status to the server This allowes

fine grain control over the update timing and useful for near real time updates

2 NCSA doesnrsquot require any daemon to be running on the client as the client pushes the update periodically

This leads to simpler install

3 NRPE does active check The server pulls the info from the client periodically This is better suited for

centrally adminsterored policies

4 NRPE requires a daemon be runnion on all the clients to service NRPE requests

5 NCSA is not in active development anymore (no updates since 2013)

Solace Nagios Toolset Integration

25

8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD

(Round Robin Database) It can collect manage and display graphs of collected data

81 Prerequisites

811 Cacti Dependencies

Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use

MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration

unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not

required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files

82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are

instructions and materials available that can be readily used Here are some lnks

httpdocscactinetmanual0881_installation1_install_unix

httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-

fedora-17-12

httpwwwcactinetdownloadsdocshtmldebugginghtml

83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to

Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for

illustration which can be replaced with any monitoring using right scripts

Solace Nagios Toolset Integration

26

831 Add Custom Data Input method

Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the

solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script

need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected

form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot

do any validation on Output Field

8311 Data Input

8312 Input

8313 Output

Solace Nagios Toolset Integration

27

832 Add Custom Data Template

Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to

declare info for the specific queue (router name vpn name queue name and credentials)

Solace Nagios Toolset Integration

28

833 Add a custom Graph Template

Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph

Template Items)

Solace Nagios Toolset Integration

29

834 Add Solace Appliance as Device

Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to

run host specific services like ping test

8341 Add Data Source List

Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown

here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the

correspoinding RRD files for these resources

Solace Nagios Toolset Integration

30

8342 Add Graph List

Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the

specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool

Solace Nagios Toolset Integration

31

84 Verification

841 Basic Install

Once you have basic Cacti setup up and running you should be able to access the install and do further configuration

thru web admin page at

httpltyour-server-ipgtcacti

After configuration you would be able to see basic system metrics for the localhost that would look similar to this

Solace Nagios Toolset Integration

32

842 Solace host check

Once Solce router is added as a device from he Devices link you would be able view the router with basic health test

such as ping status

843 Solace VPN Metrics Sample

Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue

statistics showing queue depth for a single queue on a VPN

Solace Nagios Toolset Integration

33

As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of

aggregation of queue stats over various window period from hour to year

Solace Nagios Toolset Integration

34

9 Additional Topics

91 Admin Tasks

911 Creating custom Maps

Create custom maps for any outputs that need it for example turn the iface counter values into gauge

Add to usrlocalnagiosgraphetcnagiosgraphmap

Service type iface

outputIFACE OK - [Interface161EnabledyesLinkyes]

perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-

bytes=363571175913

perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)

and push s [ if_stats

[tx-pkts COUNTER int $1 ]

[rx-pkts COUNTER int $2 ]

[tx-bytes COUNTER int $3 ]

[rx-bytes COUNTER int $4 ] ]

912 Adding additional routers VPNs

Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run

mk_nagiossolacecfg to generate required solace plugin Config files

Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be

added into etcnagiosnagioscfg file

Note that the services need to be restarted for the changes to be effective

913 Putting Nagios on Verbose mode

Change debug level in Nagios config to 2048

grep ^debug_level etcnagiosnagioscfg

debug_level=2048

Now check the debug logs All commands executed and output will be logged over there

Solace Nagios Toolset Integration

35

tail varlognagiosnagiosdebug

[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING

[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w

usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --

queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet

[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w

usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --

vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet

[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING

914 Modifying iptables

When you Cant access http from outside or when telnet to server on port 80 fail

Check Iptables routes

iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT

iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT

iptables-save | tee etcsysconfigiptables

service iptables save

service iptables restart

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

25

8 Cacti Setup Cacti is performance monitoring tool based on a LAMP stack (LinuxApacheMySQLPHP) and RRD

(Round Robin Database) It can collect manage and display graphs of collected data

81 Prerequisites

811 Cacti Dependencies

Apache A Web server to display network graphs created by PHP and RRDTool MySQLMariaDB A Database server to store cacti information Recent CentOS versions (70 including) use

MariaDB as default DB for its LAMP stack) PHP A script module to create graphs using RRDTool PHP-SNMP A PHP extension for SNMP to access data (Optional and not required for Solace integration

unless using SNMP from Solace) NET-SNMP A SNMP (Simple Network Management Protocol) is used to manage network (Optional and not

required for Solace integration unless using SNMP from Solace) RRDTool A database tool to manage and retrieve time series data like CPU load Network Bandwidth etc UnZip A simple tool for extracting zip files

82 Installation Installation of Cacti and dependencies is a topic of its own and beyond the scope of this document There are

instructions and materials available that can be readily used Here are some lnks

httpdocscactinetmanual0881_installation1_install_unix

httpwwwtecmintcominstall-cacti-network-monitoring-on-rhel-centos-6-3-5-8-and-

fedora-17-12

httpwwwcactinetdownloadsdocshtmldebugginghtml

83 Post Install Setup This section walks thru the steps to add a sample solace artifiact monitoring to Cacti using web admin Similar to

Nagios Cacti uses external scripts for gathering the monitoring info In this example Queue Depth is used for

illustration which can be replaced with any monitoring using right scripts

Solace Nagios Toolset Integration

26

831 Add Custom Data Input method

Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the

solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script

need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected

form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot

do any validation on Output Field

8311 Data Input

8312 Input

8313 Output

Solace Nagios Toolset Integration

27

832 Add Custom Data Template

Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to

declare info for the specific queue (router name vpn name queue name and credentials)

Solace Nagios Toolset Integration

28

833 Add a custom Graph Template

Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph

Template Items)

Solace Nagios Toolset Integration

29

834 Add Solace Appliance as Device

Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to

run host specific services like ping test

8341 Add Data Source List

Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown

here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the

correspoinding RRD files for these resources

Solace Nagios Toolset Integration

30

8342 Add Graph List

Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the

specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool

Solace Nagios Toolset Integration

31

84 Verification

841 Basic Install

Once you have basic Cacti setup up and running you should be able to access the install and do further configuration

thru web admin page at

httpltyour-server-ipgtcacti

After configuration you would be able to see basic system metrics for the localhost that would look similar to this

Solace Nagios Toolset Integration

32

842 Solace host check

Once Solce router is added as a device from he Devices link you would be able view the router with basic health test

such as ping status

843 Solace VPN Metrics Sample

Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue

statistics showing queue depth for a single queue on a VPN

Solace Nagios Toolset Integration

33

As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of

aggregation of queue stats over various window period from hour to year

Solace Nagios Toolset Integration

34

9 Additional Topics

91 Admin Tasks

911 Creating custom Maps

Create custom maps for any outputs that need it for example turn the iface counter values into gauge

Add to usrlocalnagiosgraphetcnagiosgraphmap

Service type iface

outputIFACE OK - [Interface161EnabledyesLinkyes]

perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-

bytes=363571175913

perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)

and push s [ if_stats

[tx-pkts COUNTER int $1 ]

[rx-pkts COUNTER int $2 ]

[tx-bytes COUNTER int $3 ]

[rx-bytes COUNTER int $4 ] ]

912 Adding additional routers VPNs

Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run

mk_nagiossolacecfg to generate required solace plugin Config files

Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be

added into etcnagiosnagioscfg file

Note that the services need to be restarted for the changes to be effective

913 Putting Nagios on Verbose mode

Change debug level in Nagios config to 2048

grep ^debug_level etcnagiosnagioscfg

debug_level=2048

Now check the debug logs All commands executed and output will be logged over there

Solace Nagios Toolset Integration

35

tail varlognagiosnagiosdebug

[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING

[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w

usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --

queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet

[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w

usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --

vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet

[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING

914 Modifying iptables

When you Cant access http from outside or when telnet to server on port 80 fail

Check Iptables routes

iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT

iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT

iptables-save | tee etcsysconfigiptables

service iptables save

service iptables restart

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

26

831 Add Custom Data Input method

Select ldquoScriptCommandrsquo as Input Type and type in the external script name that would collect the metrics from the

solace appliance Note that ltpath_cactigt is a predefined variable that can be used The input arguments for the script

need to be both declared on the ldquoInput Stringrdquo and defined in the ldquoInput Fieldsrdquo section below Similarly output expected

form the script should be defined in the ldquoOutput Fieldsrdquo section This is used mostly as a placeholder and Cacti doesnrsquot

do any validation on Output Field

8311 Data Input

8312 Input

8313 Output

Solace Nagios Toolset Integration

27

832 Add Custom Data Template

Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to

declare info for the specific queue (router name vpn name queue name and credentials)

Solace Nagios Toolset Integration

28

833 Add a custom Graph Template

Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph

Template Items)

Solace Nagios Toolset Integration

29

834 Add Solace Appliance as Device

Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to

run host specific services like ping test

8341 Add Data Source List

Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown

here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the

correspoinding RRD files for these resources

Solace Nagios Toolset Integration

30

8342 Add Graph List

Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the

specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool

Solace Nagios Toolset Integration

31

84 Verification

841 Basic Install

Once you have basic Cacti setup up and running you should be able to access the install and do further configuration

thru web admin page at

httpltyour-server-ipgtcacti

After configuration you would be able to see basic system metrics for the localhost that would look similar to this

Solace Nagios Toolset Integration

32

842 Solace host check

Once Solce router is added as a device from he Devices link you would be able view the router with basic health test

such as ping status

843 Solace VPN Metrics Sample

Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue

statistics showing queue depth for a single queue on a VPN

Solace Nagios Toolset Integration

33

As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of

aggregation of queue stats over various window period from hour to year

Solace Nagios Toolset Integration

34

9 Additional Topics

91 Admin Tasks

911 Creating custom Maps

Create custom maps for any outputs that need it for example turn the iface counter values into gauge

Add to usrlocalnagiosgraphetcnagiosgraphmap

Service type iface

outputIFACE OK - [Interface161EnabledyesLinkyes]

perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-

bytes=363571175913

perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)

and push s [ if_stats

[tx-pkts COUNTER int $1 ]

[rx-pkts COUNTER int $2 ]

[tx-bytes COUNTER int $3 ]

[rx-bytes COUNTER int $4 ] ]

912 Adding additional routers VPNs

Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run

mk_nagiossolacecfg to generate required solace plugin Config files

Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be

added into etcnagiosnagioscfg file

Note that the services need to be restarted for the changes to be effective

913 Putting Nagios on Verbose mode

Change debug level in Nagios config to 2048

grep ^debug_level etcnagiosnagioscfg

debug_level=2048

Now check the debug logs All commands executed and output will be logged over there

Solace Nagios Toolset Integration

35

tail varlognagiosnagiosdebug

[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING

[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w

usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --

queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet

[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w

usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --

vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet

[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING

914 Modifying iptables

When you Cant access http from outside or when telnet to server on port 80 fail

Check Iptables routes

iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT

iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT

iptables-save | tee etcsysconfigiptables

service iptables save

service iptables restart

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

27

832 Add Custom Data Template

Using the Data Input method defined above define a data template for a specifc queue Use New Data Souce Item to

declare info for the specific queue (router name vpn name queue name and credentials)

Solace Nagios Toolset Integration

28

833 Add a custom Graph Template

Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph

Template Items)

Solace Nagios Toolset Integration

29

834 Add Solace Appliance as Device

Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to

run host specific services like ping test

8341 Add Data Source List

Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown

here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the

correspoinding RRD files for these resources

Solace Nagios Toolset Integration

30

8342 Add Graph List

Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the

specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool

Solace Nagios Toolset Integration

31

84 Verification

841 Basic Install

Once you have basic Cacti setup up and running you should be able to access the install and do further configuration

thru web admin page at

httpltyour-server-ipgtcacti

After configuration you would be able to see basic system metrics for the localhost that would look similar to this

Solace Nagios Toolset Integration

32

842 Solace host check

Once Solce router is added as a device from he Devices link you would be able view the router with basic health test

such as ping status

843 Solace VPN Metrics Sample

Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue

statistics showing queue depth for a single queue on a VPN

Solace Nagios Toolset Integration

33

As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of

aggregation of queue stats over various window period from hour to year

Solace Nagios Toolset Integration

34

9 Additional Topics

91 Admin Tasks

911 Creating custom Maps

Create custom maps for any outputs that need it for example turn the iface counter values into gauge

Add to usrlocalnagiosgraphetcnagiosgraphmap

Service type iface

outputIFACE OK - [Interface161EnabledyesLinkyes]

perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-

bytes=363571175913

perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)

and push s [ if_stats

[tx-pkts COUNTER int $1 ]

[rx-pkts COUNTER int $2 ]

[tx-bytes COUNTER int $3 ]

[rx-bytes COUNTER int $4 ] ]

912 Adding additional routers VPNs

Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run

mk_nagiossolacecfg to generate required solace plugin Config files

Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be

added into etcnagiosnagioscfg file

Note that the services need to be restarted for the changes to be effective

913 Putting Nagios on Verbose mode

Change debug level in Nagios config to 2048

grep ^debug_level etcnagiosnagioscfg

debug_level=2048

Now check the debug logs All commands executed and output will be logged over there

Solace Nagios Toolset Integration

35

tail varlognagiosnagiosdebug

[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING

[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w

usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --

queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet

[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w

usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --

vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet

[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING

914 Modifying iptables

When you Cant access http from outside or when telnet to server on port 80 fail

Check Iptables routes

iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT

iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT

iptables-save | tee etcsysconfigiptables

service iptables save

service iptables restart

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

28

833 Add a custom Graph Template

Now create a Graph Template and associate the Data Template defined earlier with the graph template (using Graph

Template Items)

Solace Nagios Toolset Integration

29

834 Add Solace Appliance as Device

Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to

run host specific services like ping test

8341 Add Data Source List

Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown

here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the

correspoinding RRD files for these resources

Solace Nagios Toolset Integration

30

8342 Add Graph List

Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the

specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool

Solace Nagios Toolset Integration

31

84 Verification

841 Basic Install

Once you have basic Cacti setup up and running you should be able to access the install and do further configuration

thru web admin page at

httpltyour-server-ipgtcacti

After configuration you would be able to see basic system metrics for the localhost that would look similar to this

Solace Nagios Toolset Integration

32

842 Solace host check

Once Solce router is added as a device from he Devices link you would be able view the router with basic health test

such as ping status

843 Solace VPN Metrics Sample

Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue

statistics showing queue depth for a single queue on a VPN

Solace Nagios Toolset Integration

33

As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of

aggregation of queue stats over various window period from hour to year

Solace Nagios Toolset Integration

34

9 Additional Topics

91 Admin Tasks

911 Creating custom Maps

Create custom maps for any outputs that need it for example turn the iface counter values into gauge

Add to usrlocalnagiosgraphetcnagiosgraphmap

Service type iface

outputIFACE OK - [Interface161EnabledyesLinkyes]

perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-

bytes=363571175913

perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)

and push s [ if_stats

[tx-pkts COUNTER int $1 ]

[rx-pkts COUNTER int $2 ]

[tx-bytes COUNTER int $3 ]

[rx-bytes COUNTER int $4 ] ]

912 Adding additional routers VPNs

Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run

mk_nagiossolacecfg to generate required solace plugin Config files

Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be

added into etcnagiosnagioscfg file

Note that the services need to be restarted for the changes to be effective

913 Putting Nagios on Verbose mode

Change debug level in Nagios config to 2048

grep ^debug_level etcnagiosnagioscfg

debug_level=2048

Now check the debug logs All commands executed and output will be logged over there

Solace Nagios Toolset Integration

35

tail varlognagiosnagiosdebug

[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING

[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w

usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --

queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet

[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w

usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --

vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet

[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING

914 Modifying iptables

When you Cant access http from outside or when telnet to server on port 80 fail

Check Iptables routes

iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT

iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT

iptables-save | tee etcsysconfigiptables

service iptables save

service iptables restart

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

29

834 Add Solace Appliance as Device

Now add each appliance as a device so graphs for this appliance can all be grouped under here This will also allow to

run host specific services like ping test

8341 Add Data Source List

Use ldquoData Souce Listrdquo link above the Device screen and using ldquoAddrdquo link add the the endpoint stats here Also shown

here is another data source for ping latency for the appliance These steps will ensure Cacti will generate the

correspoinding RRD files for these resources

Solace Nagios Toolset Integration

30

8342 Add Graph List

Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the

specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool

Solace Nagios Toolset Integration

31

84 Verification

841 Basic Install

Once you have basic Cacti setup up and running you should be able to access the install and do further configuration

thru web admin page at

httpltyour-server-ipgtcacti

After configuration you would be able to see basic system metrics for the localhost that would look similar to this

Solace Nagios Toolset Integration

32

842 Solace host check

Once Solce router is added as a device from he Devices link you would be able view the router with basic health test

such as ping status

843 Solace VPN Metrics Sample

Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue

statistics showing queue depth for a single queue on a VPN

Solace Nagios Toolset Integration

33

As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of

aggregation of queue stats over various window period from hour to year

Solace Nagios Toolset Integration

34

9 Additional Topics

91 Admin Tasks

911 Creating custom Maps

Create custom maps for any outputs that need it for example turn the iface counter values into gauge

Add to usrlocalnagiosgraphetcnagiosgraphmap

Service type iface

outputIFACE OK - [Interface161EnabledyesLinkyes]

perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-

bytes=363571175913

perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)

and push s [ if_stats

[tx-pkts COUNTER int $1 ]

[rx-pkts COUNTER int $2 ]

[tx-bytes COUNTER int $3 ]

[rx-bytes COUNTER int $4 ] ]

912 Adding additional routers VPNs

Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run

mk_nagiossolacecfg to generate required solace plugin Config files

Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be

added into etcnagiosnagioscfg file

Note that the services need to be restarted for the changes to be effective

913 Putting Nagios on Verbose mode

Change debug level in Nagios config to 2048

grep ^debug_level etcnagiosnagioscfg

debug_level=2048

Now check the debug logs All commands executed and output will be logged over there

Solace Nagios Toolset Integration

35

tail varlognagiosnagiosdebug

[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING

[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w

usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --

queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet

[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w

usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --

vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet

[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING

914 Modifying iptables

When you Cant access http from outside or when telnet to server on port 80 fail

Check Iptables routes

iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT

iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT

iptables-save | tee etcsysconfigiptables

service iptables save

service iptables restart

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

30

8342 Add Graph List

Using ldquoGraph Listrdquo link above the device screen add new graph link by clicking ldquoNewrdquo link Link the Data Source for the

specific queue created earlier This step will ensure graphs are generated from the RRD file using rrdgraph tool

Solace Nagios Toolset Integration

31

84 Verification

841 Basic Install

Once you have basic Cacti setup up and running you should be able to access the install and do further configuration

thru web admin page at

httpltyour-server-ipgtcacti

After configuration you would be able to see basic system metrics for the localhost that would look similar to this

Solace Nagios Toolset Integration

32

842 Solace host check

Once Solce router is added as a device from he Devices link you would be able view the router with basic health test

such as ping status

843 Solace VPN Metrics Sample

Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue

statistics showing queue depth for a single queue on a VPN

Solace Nagios Toolset Integration

33

As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of

aggregation of queue stats over various window period from hour to year

Solace Nagios Toolset Integration

34

9 Additional Topics

91 Admin Tasks

911 Creating custom Maps

Create custom maps for any outputs that need it for example turn the iface counter values into gauge

Add to usrlocalnagiosgraphetcnagiosgraphmap

Service type iface

outputIFACE OK - [Interface161EnabledyesLinkyes]

perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-

bytes=363571175913

perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)

and push s [ if_stats

[tx-pkts COUNTER int $1 ]

[rx-pkts COUNTER int $2 ]

[tx-bytes COUNTER int $3 ]

[rx-bytes COUNTER int $4 ] ]

912 Adding additional routers VPNs

Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run

mk_nagiossolacecfg to generate required solace plugin Config files

Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be

added into etcnagiosnagioscfg file

Note that the services need to be restarted for the changes to be effective

913 Putting Nagios on Verbose mode

Change debug level in Nagios config to 2048

grep ^debug_level etcnagiosnagioscfg

debug_level=2048

Now check the debug logs All commands executed and output will be logged over there

Solace Nagios Toolset Integration

35

tail varlognagiosnagiosdebug

[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING

[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w

usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --

queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet

[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w

usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --

vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet

[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING

914 Modifying iptables

When you Cant access http from outside or when telnet to server on port 80 fail

Check Iptables routes

iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT

iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT

iptables-save | tee etcsysconfigiptables

service iptables save

service iptables restart

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

31

84 Verification

841 Basic Install

Once you have basic Cacti setup up and running you should be able to access the install and do further configuration

thru web admin page at

httpltyour-server-ipgtcacti

After configuration you would be able to see basic system metrics for the localhost that would look similar to this

Solace Nagios Toolset Integration

32

842 Solace host check

Once Solce router is added as a device from he Devices link you would be able view the router with basic health test

such as ping status

843 Solace VPN Metrics Sample

Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue

statistics showing queue depth for a single queue on a VPN

Solace Nagios Toolset Integration

33

As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of

aggregation of queue stats over various window period from hour to year

Solace Nagios Toolset Integration

34

9 Additional Topics

91 Admin Tasks

911 Creating custom Maps

Create custom maps for any outputs that need it for example turn the iface counter values into gauge

Add to usrlocalnagiosgraphetcnagiosgraphmap

Service type iface

outputIFACE OK - [Interface161EnabledyesLinkyes]

perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-

bytes=363571175913

perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)

and push s [ if_stats

[tx-pkts COUNTER int $1 ]

[rx-pkts COUNTER int $2 ]

[tx-bytes COUNTER int $3 ]

[rx-bytes COUNTER int $4 ] ]

912 Adding additional routers VPNs

Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run

mk_nagiossolacecfg to generate required solace plugin Config files

Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be

added into etcnagiosnagioscfg file

Note that the services need to be restarted for the changes to be effective

913 Putting Nagios on Verbose mode

Change debug level in Nagios config to 2048

grep ^debug_level etcnagiosnagioscfg

debug_level=2048

Now check the debug logs All commands executed and output will be logged over there

Solace Nagios Toolset Integration

35

tail varlognagiosnagiosdebug

[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING

[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w

usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --

queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet

[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w

usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --

vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet

[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING

914 Modifying iptables

When you Cant access http from outside or when telnet to server on port 80 fail

Check Iptables routes

iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT

iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT

iptables-save | tee etcsysconfigiptables

service iptables save

service iptables restart

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

32

842 Solace host check

Once Solce router is added as a device from he Devices link you would be able view the router with basic health test

such as ping status

843 Solace VPN Metrics Sample

Once VPN artifiact checkings are added you should be able to view them under Graphs tab Here is a sample of queue

statistics showing queue depth for a single queue on a VPN

Solace Nagios Toolset Integration

33

As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of

aggregation of queue stats over various window period from hour to year

Solace Nagios Toolset Integration

34

9 Additional Topics

91 Admin Tasks

911 Creating custom Maps

Create custom maps for any outputs that need it for example turn the iface counter values into gauge

Add to usrlocalnagiosgraphetcnagiosgraphmap

Service type iface

outputIFACE OK - [Interface161EnabledyesLinkyes]

perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-

bytes=363571175913

perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)

and push s [ if_stats

[tx-pkts COUNTER int $1 ]

[rx-pkts COUNTER int $2 ]

[tx-bytes COUNTER int $3 ]

[rx-bytes COUNTER int $4 ] ]

912 Adding additional routers VPNs

Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run

mk_nagiossolacecfg to generate required solace plugin Config files

Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be

added into etcnagiosnagioscfg file

Note that the services need to be restarted for the changes to be effective

913 Putting Nagios on Verbose mode

Change debug level in Nagios config to 2048

grep ^debug_level etcnagiosnagioscfg

debug_level=2048

Now check the debug logs All commands executed and output will be logged over there

Solace Nagios Toolset Integration

35

tail varlognagiosnagiosdebug

[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING

[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w

usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --

queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet

[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w

usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --

vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet

[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING

914 Modifying iptables

When you Cant access http from outside or when telnet to server on port 80 fail

Check Iptables routes

iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT

iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT

iptables-save | tee etcsysconfigiptables

service iptables save

service iptables restart

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

33

As with any other Cacti graph aggrecation over a peroid is available by clicking on the graph Here is an example of

aggregation of queue stats over various window period from hour to year

Solace Nagios Toolset Integration

34

9 Additional Topics

91 Admin Tasks

911 Creating custom Maps

Create custom maps for any outputs that need it for example turn the iface counter values into gauge

Add to usrlocalnagiosgraphetcnagiosgraphmap

Service type iface

outputIFACE OK - [Interface161EnabledyesLinkyes]

perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-

bytes=363571175913

perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)

and push s [ if_stats

[tx-pkts COUNTER int $1 ]

[rx-pkts COUNTER int $2 ]

[tx-bytes COUNTER int $3 ]

[rx-bytes COUNTER int $4 ] ]

912 Adding additional routers VPNs

Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run

mk_nagiossolacecfg to generate required solace plugin Config files

Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be

added into etcnagiosnagioscfg file

Note that the services need to be restarted for the changes to be effective

913 Putting Nagios on Verbose mode

Change debug level in Nagios config to 2048

grep ^debug_level etcnagiosnagioscfg

debug_level=2048

Now check the debug logs All commands executed and output will be logged over there

Solace Nagios Toolset Integration

35

tail varlognagiosnagiosdebug

[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING

[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w

usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --

queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet

[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w

usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --

vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet

[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING

914 Modifying iptables

When you Cant access http from outside or when telnet to server on port 80 fail

Check Iptables routes

iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT

iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT

iptables-save | tee etcsysconfigiptables

service iptables save

service iptables restart

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

34

9 Additional Topics

91 Admin Tasks

911 Creating custom Maps

Create custom maps for any outputs that need it for example turn the iface counter values into gauge

Add to usrlocalnagiosgraphetcnagiosgraphmap

Service type iface

outputIFACE OK - [Interface161EnabledyesLinkyes]

perfdate rx-pkts=365832598 rx-bytes=108312525627 tx-pkts=390410814 tx-

bytes=363571175913

perfdatarx-pkts=(d+) rx-bytes=(d+) tx-pkts=(d+) tx-bytes=(d+)

and push s [ if_stats

[tx-pkts COUNTER int $1 ]

[rx-pkts COUNTER int $2 ]

[tx-bytes COUNTER int $3 ]

[rx-bytes COUNTER int $4 ] ]

912 Adding additional routers VPNs

Use cfgsampleroutercfg as template to create additional router VPN or VPN bridge configuration Run

mk_nagiossolacecfg to generate required solace plugin Config files

Alternatively the files can be edited or added in etcnagiosobjects dir manually The corresponding entries need to be

added into etcnagiosnagioscfg file

Note that the services need to be restarted for the changes to be effective

913 Putting Nagios on Verbose mode

Change debug level in Nagios config to 2048

grep ^debug_level etcnagiosnagioscfg

debug_level=2048

Now check the debug logs All commands executed and output will be logged over there

Solace Nagios Toolset Integration

35

tail varlognagiosnagiosdebug

[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING

[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w

usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --

queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet

[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w

usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --

vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet

[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING

914 Modifying iptables

When you Cant access http from outside or when telnet to server on port 80 fail

Check Iptables routes

iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT

iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT

iptables-save | tee etcsysconfigiptables

service iptables save

service iptables restart

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

35

tail varlognagiosnagiosdebug

[1446670816003569] [20481] [pid=25423] BEGIN MACRO PROCESSING

[1446670816003572] [20481] [pid=25423] Processing usrbinperl -w

usrlocalnagiospluginsqueuepl --router $ARG1$ --user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --

queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --checkAvail --quiet

[1446670816003584] [20481] [pid=25423] Done Final output usrbinperl -w

usrlocalnagiospluginsqueuepl --router 192168561038080 --user admin --pass admin --

vpn vpn2 --queue q5 --warning 100 --critical 1000 --checkAvail --quiet

[1446670816003595] [20481] [pid=25423] END MACRO PROCESSING

914 Modifying iptables

When you Cant access http from outside or when telnet to server on port 80 fail

Check Iptables routes

iptables -I INPUT 5 -i eth0 -p tcp --dport 80 -m state --state NEWESTABLISHED -j ACCEPT

iptables -I INPUT 5 -i eth0 -p tcp --dport 443 -m state --state NEWESTABLISHED -j ACCEPT

iptables-save | tee etcsysconfigiptables

service iptables save

service iptables restart

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

36

92 Troubleshooting

921 Nagios not starting up

Check Config file for any errors

usrsbinnagios -v etcnagiosnagioscfg

922 Canrsquot connect to HTTP (or other port) from outside

When access to port is working form within the box (for localhost) and not form outside check the iptables

923 CGI Error in showcgi

Check http errorlog

[Fri May 02 022112 2014] [error] [client 1921681501] (13)Permission denied exec of

usrlocalnagiossbinshowcgi failed referer http192168150137nagioscgi-

binstatuscgihost=all

[Fri May 02 022112 2014] [error] [client 1921681501] Premature end of script headers

showcgi referer http192168150137nagioscgi-binstatuscgihost=all

Check file and dir permissions They should be world readable and executable

Check SE (Security Enhanced Linux) Mode

chmod go+rx usrlocalnagiossbin

chmod go+rx usrlocalnagiossbinshowcgi

setenforce 0

service httpd restart

service nagios restart

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

37

924 CGI Error in Solace graphs

Check that the CGI scripts are runnable from command line

Check for missing required modules and Perl libraries

usrlocalnagiospluginsdiskpl -r solace-routerport -u user -p password -v

DISK OK - Disk 1 [State up Enabled true] Disk 2 [State up Enabled true] RAID [in

fully redundant state]

925 Canrsquot stat command file error

If you see error

Error Could not stat() command file lsquousrlocalnagiosvarrwnagioscmdrsquo

chcon -R -t httpd_sys_script_rw_t usrlocalnagiosvarrw (change path based on the error message)

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

38

10 Setup Scripts and Config files This section lists sample config files and setup scripts used in this document Please check respected product

document for most recent syntax for config files

101 Install Info Default install paths and files

HTTP (Apache)

Config etchttpdconfhttpdconf

Logs varloghttpderror_log varloghttpdaccess_log

Nagios

Config file etcnagiosnagioscfg

Commands etcnagiosobjectscommandscfg

Log varlognagiosnagioslog

Perfdata log varlognagiosperfdatalog

Config Dir etcnagiosobjects

Command file varspoolnagioscmdnagioscmd

RRD

Data dir varnagiosrrd

Nagiosgraph

Config usrlocalnagiosgraphetc nagiosgraphconf

Apache config usrlocalnagiosgraphetc nagiosgraph-apacheconf

Log usrlocalnagiosgraphvarnagiosgraphlog

Nagios Solace

Config etcnagiossolacesolacecfg

Plugin dir usrlocalnagiosplugins

NSCA Daemon and Agent

Path usrlocalnsca

Config (server) usrlocalnscacfgnscacfg

Config (client) usrlocalnscacfgnsca_servercfg

usrlocalnscacfgnsca_clientcfg

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

39

102 Nagios This section lists minimal files required for Solace Nagios setup Pl contact Solace PSG support to get a tarball

package with the files and required plugin files

1021 setup_nagiossolace

binbash

setup_nagiossolace

httpcfg=etchttpdconfhttpdconf

nagioscfg=etcnagiosnagioscfg

nagiosgraphcfg=usrlocalnagiosgraphetcnagiosgraphconf

nagiosgraphapachecfg=usrlocalnagiosgraphetcnagiosgraph-apacheconf

nagioscmdfile=etcnagiosobjectscommandscfg

nagiostmplfile=etcnagiosobjectstemplatescfg

nagioslocalhostcfg=etcnagiosobjectslocalhostcfg

nagiosgraphcmdfile=usrlocalnagiosgraphetcnagiosgraph-commandscfg

nagiosgraphinsertscript=usrlocalnagiosgraphbininsertpl

nagiossolaceplugindir=usrlocalnagiosplugins

nagiossolacecfgdir=etcnagiossolace

tag=BEGIN_SETUP-Nagiosgraph

echo

echo Checking required files

for f in $httpcfg $nagioscfg $nagiosgraphcfg $nagiosgraphapachecfg $nagioscmdfile $nagiostmplfile $nagioslocalhostcfg

$nagiosgraphinsertscript do

if [ -f $f ] then

echo $f missing Exiting

exit

fi

echo $f

grep -q $tag $f ampamp echo already setup will be skipped

done

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

40

-------------------------------------------------------------------------

ts=$(date +ymd_HMS)

echo

echo Setting up config files

echo $nagioscfg

grep -q $tag $nagioscfg ||

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt $nagioscfg ltlt__END__

--- $tag

process nagios performance data using nagiosgraph

process_performance_data=1

service_perfdata_file=varlognagiosperfdatalog

service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=30

service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph

host config with sample nagiosgraphs

cfg_file=etcnagiosobjectslocalhost-graphcfg

solace config

cfg_file=etcnagiossolacesolacecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo $nagiostmplfile

grep -q $tag $nagiostmplfile ||

cp -p $nagiostmplfile $nagiostmplfilesav-$ts

cat gtgt $nagiostmplfile ltlt__END__

-- $tag

define service

name graphed-service

action_url nagiosgraphcgi-binshowcgihost=$HOSTNAME$ampservice=$SERVICEDESC$

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

41

onMouseOver=showGraphPopup(this) onMouseOut=hideGraphPopup() rel=nagiosgraphcgi-

binshowgraphcgihost=$HOSTNAME$ampservice=$SERVICEDESC$ampperiod=weekamprrdopts=-w+450+-j

register 0

-- end setup

__END__

-------------------------------------------------------------------------

localhost-graphcfg generaed if not there already

outfile=$(dirname $nagioslocalhostcfg)localhost-graphcfg

echo $outfile

if [ -f $outfile ] then

sed -e slocal-servicelocal-servicegraphed-serviceg

-e s(localhost)1-grg

-e s(linux-servers)1-grg

$nagioslocalhostcfg gt tmplocalhost-graphcfg

cp -p tmplocalhost-graphcfg $outfile

fi

-------------------------------------------------------------------------

echo $nagioscmdfile

grep -q $tag $nagioscmdfile ||

cp -p $nagioscmdfile $nagioscmdfilesav-$ts

cat gtgt $nagioscmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata-for-nagiosgraph

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

42

echo $nagiosgraphcmdfile

grep -q $tag $nagiosgraphcmdfile ||

cp -p $nagiosgraphcmdfile $nagiosgraphcmdfilesav-$ts

cat gtgt $nagiosgraphcmdfile ltlt__END__

-- $tag

command to process nagios performance data for nagiosgraph

define command

command_name process-service-perfdata

command_line $nagiosgraphinsertscript

-- end setup

__END__

-------------------------------------------------------------------------

note -- this file is overwritten (not append)

echo $nagiosgraphapachecfg

grep -q $tag $nagiosgraphapachecfg ||

cp -p $nagiosgraphapachecfg $nagiosgraphapachecfgsav-$ts

cat gt $nagiosgraphapachecfg ltlt__END__

--- $tag

enable nagiosgraph CGI scripts

ScriptAlias nagiosgraphcgi-bin usrlocalnagiosgraphcgi

ltDirectory usrlocalnagiosgraphcgigt

Options ExecCGI

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

enable nagiosgraph CSS and JavaScript

Alias nagiosgraph usrlocalnagiosgraphshare

ltDirectory usrlocalnagiosgraphsharegt

Options None

AllowOverride None

Order allowdeny

Allow from all

ltDirectorygt

-- end setup

__END__

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

43

-------------------------------------------------------------------------

echo $nagiosgraphcfg

grep -q $tag $nagiosgraphcfg ||

cp -p $nagiosgraphcfg $nagiosgraphcfgsav-$ts

sed -e s(nagiosgraphcgiurl =)()-- $tagn12n1 nagiosgraphcgi-binn--end setupng

$nagiosgraphcfg gt tmpnagiosgraphcfg

sed -e snagioscgi-binnagiosgraphcgi-bing

-e snagiosnagiosgraphjsnagiosgraphnagiosgraphjsg

-e snagiosnagiosgraphcssnagiosgraphnagiosgraphcssg

$nagiosgraphcfg gt tmpnagiosgraphcfg

cp -p tmpnagiosgraphcfg $nagiosgraphcfg

-------------------------------------------------------------------------

echo $httpcfg

grep -q $tag $httpcfg ||

cp -p $httpcfg $httpcfgsav-$ts

cat gtgt $httpcfg ltlt__END__

--- $tag

include $nagiosgraphapachecfg

-- end setup

__END__

-------------------------------------------------------------------------

echo

echo Setting Solace for Nagios

if [ -d plugin ] then

echo $nagiossolaceplugindir

[ -d $nagiossolaceplugindir ] || mkdir -p $nagiossolaceplugindir

cp -p plugin $nagiossolaceplugindir

chmod 555 $nagiossolaceplugindir

else

echo No plugin dir in package $nagiossolaceplugindir not setup

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

44

fi

if [ -f cfgsolacecfg ] then

echo $nagiossolacecfgdir

[ -d $nagiossolacecfgdir ] || mkdir -p $nagiossolacecfgdir

cp -p cfgsolacecfg $nagiossolacecfgdir

else

echo No cfg file cfgsolacecfg $nagiossolacecfgdir not setup

fi

-------------------------------------------------------------------------

try to gernerate sample cfg

mk_nagiossolacecfg cfgsampleroutercfg

-------------------------------------------------------------------------

echo

echo Restarting services

for s in nagios httpd do

echo $s

systemctl restart $s gt tmp$sout 2gtamp1

systemctl status $s

done

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

45

1022 mk_nagiossolacecfg

binbash

mk_nagiossolacecfg

generate nagios cfg file solace router from template

History

-------------------------------------------------------------

nram Nov 04 2015

- check and exit if cfgfile is added in nagioscfg

- handle cfg file with no or only unidirectional bridge

me=mk_nagiossolacecfg

nagioscfg=etcnagiosnagioscfg

nagioscmdfile=etcnagiosobjectscommandscfg

echo

echo Generating Nagios config for Solace appliance

tmplfile=cfgsolace-templateroutercfg

if [ $ -lt 1 ] then

echo Missing cfgfile argument exiting

exit

fi

cfgfile=$1

if [ -f $cfgfile ] then

echo Bad cfgfile argument $cfgfile exiting

exit

fi

echo Using cfgfile $cfgfile

$cfgfile

echo VPN_BRIDGENAME lt$VPN_BRIDGENAMEgt

echo VPN_BRIDGENAME_BI lt$VPN_BRIDGENAME_BIgt

if [ -z $VPN_BRIDGENAME ] then

tmplfile=cfgsolace-templaterouter-no_bridgecfg

elif [ -z $VPN_BRIDGENAME_BI ] then

tmplfile=cfgsolace-templaterouter-no_bi_bridgecfg

fi

if [ -f $tmplfile ] then

echo Template file not found $tmplfile exiting

exit

fi

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

46

echo Using template file $tmplfile

ext=$outfile

fname=$(basename $cfgfile)

outfile=etcnagiosobjectssolace-$fnamecfg

if [ -f $outfile ] then

echo File $outfile already exists exiting

exit

fi

grep -q ^cfg_file=$outfile $nagioscfg ampamp

echo File $outfile already in $nagioscfg exiting

exit

gt $outfile ||

echo Cant create output file $outfile exiting

exit

echo Generating config

echo $outfile

cat gt $outfile ltlt__END__

Nagios config for soalce appliance

generated from cfg file $cfgfile

template file used $tmplfile

$(date)

__END__

cat $tmplfile | while read line

do

eval echo $line

done gtgt $outfile

ts=$(date +ymd_HMS)

echo Setting up config files

echo $nagioscfg

cp -p $nagioscfg $nagioscfgsav-$ts

cat gtgt$nagioscfg ltlt__END__

-- created by $me $(date +D T)

cfg_file=$outfile

__END__

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

47

1023 Solace Nagios Config (cfgsolacecfg)

define command

command_name info_not_defined

command_line usrbinperl -w usrlocalnagiospluginsinfopl -s INFO -m Not

defined in config

define command

command_name alarm

command_line usrbinperl -w usrlocalnagiospluginsalarmpl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name bridge_uni

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -q

define command

command_name bridge_bi

command_line usrbinperl -w usrlocalnagiospluginsbridgepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -b $ARG4$ --vpn $ARG5$ -o -q

define command

command_name disk

command_line usrbinperl -w usrlocalnagiospluginsdiskpl -r $ARG1$ -u $ARG2$

-p $ARG3$

define command

command_name env

command_line usrbinperl -w usrlocalnagiospluginsenvironmentpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ --wf $ARG4$ --cf $ARG5$ --wc $ARG6$ --cc $ARG7$ --wn $ARG8$ --cn $ARG9$

define command

command_name hardware

command_line usrbinperl -w usrlocalnagiospluginshardwarepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -a -b

define command

command_name iface_eth

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

48

$ARG2$ -p $ARG3$ -i $ARG4$

define command

command_name iface_lag

command_line usrbinperl -w usrlocalnagiospluginsifacepl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -i $ARG4$ --wa $ARG5$ --ca $ARG6$ --wo $ARG7$ --co $ARG8$

define command

command_name memory

command_line usrbinperl -w usrlocalnagiospluginsmemorypl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$

define command

command_name spool_vpn

command_line usrbinperl -w usrlocalnagiospluginsmessage-spool_vpnpl -r

$ARG1$ -u $ARG2$ -p $ARG3$ --vpn $ARG4$ -w $ARG5$ -c $ARG6$

define command

command_name message-spool

command_line usrbinperl -w usrlocalnagiospluginsmessage-spoolpl -r $ARG1$

-u $ARG2$ -p $ARG3$ --wt $ARG4$ --ct $ARG5$ --waf $ARG6$ --caf $ARG7$ --wam $ARG8$ --cam

$ARG9$ --wdf $ARG10$ --cdf $ARG11$ --wdm $ARG12$ --cdm $ARG13$ --wtf $ARG14$ --ctf $ARG15$ -

-wtm $ARG16$ --ctm $ARG17$ --quiet

define command

command_name partition

command_line usrbinperl -w usrlocalnagiospluginspartitionpl -r $ARG1$ -u

$ARG2$ -p $ARG3$ -w $ARG4$ -c $ARG5$ -f $ARG6$

define command

command_name queue

command_line usrbinperl -w usrlocalnagiospluginsqueuepl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --queue $ARG5$ --warning $ARG6$ --critical $ARG7$ --

checkAvail --quiet

define command

command_name redundancy

command_line usrbinperl -w usrlocalnagiospluginsredundancypl -r $ARG1$ -u

$ARG2$ -p $ARG3$

define command

command_name replication

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

49

command_line usrbinperl -w usrlocalnagiospluginsreplicationpl -r $ARG1$ -

u $ARG2$ -p $ARG3$ --vpn $ARG4$ --quiet

define command

command_name vpn

command_line usrbinperl -w usrlocalnagiospluginsvpnpl --router $ARG1$ --

user $ARG2$ --pass $ARG3$ --vpn $ARG4$ --warnInRate $ARG5$ --critInRate $ARG6$ --

warnOutRate $ARG7$ --critOutRate $ARG8$ --warnConn $ARG9$ --critConn $ARG10$ --checkAvail

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

50

1024 Template router (cfgsolace-templateroutercfg)

define host

use generic-host

host_name $HOSTNAME

alias $HOSTNAME

address $HOSTIP

max_check_attempts 10

define service

use generic-service

host_name $HOSTNAME

service_description alarm

check_command alarm$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_uni$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description bridge

check_command

bridge_bi$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPN_BRIDGENAME_BI$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description disk

check_command disk$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description env

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

51

check_command

env$HOSTIP$HOSTPORT$USERNAME$PASSWORD20001000809090100

define service

use generic-service

host_name $HOSTNAME

service_description hardware

check_command hardware$HOSTIP$HOSTPORT$USERNAME$PASSWORD21

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface

check_command iface_eth$HOSTIP$HOSTPORT$USERNAME$PASSWORD161

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description iface_l

check_command

iface_lag$HOSTIP$HOSTPORT$USERNAME$PASSWORD16lag11111

define service

use generic-service

host_name $HOSTNAME

service_description memory

check_command memory$HOSTIP$HOSTPORT$USERNAME$PASSWORD8090

define service

use generic-service

host_name $HOSTNAME

service_description spool_vpn

check_command

spool_vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME7090

define service

use generic-service

host_name $HOSTNAME

service_description message-spool

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

52

check_command message-

spool$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090200020002000020000300005500010000005000

00003000055000100000050000000

define service

use generic-service

host_name $HOSTNAME

service_description partition

check_command

partition$HOSTIP$HOSTPORT$USERNAME$PASSWORD7090devmd6

define service

use generic-servicegraphed-service

host_name $HOSTNAME

service_description queue

check_command

queue$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAMEtest1001000

define service

use generic-service

host_name $HOSTNAME

service_description redundancy

check_command redundancy$HOSTIP$HOSTPORT$USERNAME$PASSWORD

define service

use generic-service

host_name $HOSTNAME

service_description replication

check_command

replication$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME

define service

use generic-service

host_name $HOSTNAME

service_description vpn

check_command

vpn$HOSTIP$HOSTPORT$USERNAME$PASSWORD$VPNNAME1000100001000010000050500

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

53

1025 Template and Sample router Config

$ cat cfgsampleroutercfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=your_hostname

HOSTIP=your_hostip

HOSTPORT=your_hostport

USERNAME=user_username

PASSWORD=user_password

VPNNAME=your_vpn

VPN_BRIDGENAME=your_bridgename-uni

VPN_BRIDGENAME_BI=your_bridgename-bi

$ cat cfgvmr2cfg

change the your_XXX in this file before running setup_nagiossolace script

HOSTNAME=solace-vmr2

HOSTIP=1921681140

HOSTPORT=8080

USERNAME=admin

PASSWORD=admin

VPNNAME=vpn1

VPN_BRIDGENAME=bridge_vpn1_to_2

VPN_BRIDGENAME_BI=

103 Solace Nagios Plugin Please contact Solace PSG Support for getting copy of Solace plugin files Please refer to this link for contact info

httpdevsolacesystemscomcontact-us

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

54

104 NSCA

1041 setup_nsca

binbash

setup_nsca setup nsca daemon

while getopts st arg

do

case $arg in

s) SRCDIR=$OPTARG

i) SVRIP=$OPTARG

t) TARGETDIR=$OPTARG

) printf Usage s -a ltnsca-src-dirgt [-t ltinstall-target-dirgt]n $0

exit 2

esac

done

if [ -z $SRCDIR ] then

printf Missing -s ltnsca-src-dirgt Exiting n

exit 2

fi

printf srcdir $SRCDIR okn

if [ -z $SVRIP ] then

SVRIP=127001

printf Missing -i ltserver-ipgt Using $SVRIPn

fi

printf serverip $SVRIPn

[ -z $TARGETDIR ] ampamp TARGETDIR=usrlocalnsca

if [ -d $TARGETDIR ] then

printf target dir $TARGETDIR already exists Exiting n

exit 3

fi

printf target dir $TARGETDIR okn

echo

host=$SVRIP

port=5667

gt cfgnsca_servercfg

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

55

printf Checking n

for file in srcnsca srcsend_nsca do

if [ -x $SRCDIR$file ] then

printf Required file $SRCDIR$file not found Exiting n

exit 3

fi

printf file $file okn

done

for file in scriptsnscainit scriptssend_ngalertpl cfgnscacfg cfgnsca_servercfg

cfgnsca_clientcfg do

if [ -f $file ] then

printf Required file $file not found Exiting n

exit 3

fi

printf file $file okn

done

mkdir -p $TARGETDIRbincfgsbin

[ -d varrunnsca ] || mkdir varrunnsca

printf Installing n

printf $TARGETDIRsbinnsca n

cp -p $SRCDIRsrcnsca $TARGETDIRsbin

printf $TARGETDIRbinsend_nsca n

cp -p $SRCDIRsrcsend_nsca $TARGETDIRbin

printf $TARGETDIRcfgnscacfg n

cp -p cfgnscacfg $TARGETDIRcfg

printf $TARGETDIRbinsend_ngalertpl n

cp -p scriptssend_ngalertpl $TARGETDIRbin

if [ -f etcinitdnsca ] then

printf etcinitdnsca n

cp -p scriptsnscainit etcinitdnsca

printf Creating service n

chkconfig --add nsca

else

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

56

echo etcinitdnsca exists Skipping service creation

fi

printf Installation completen

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

57

1042 send_ngalertpl

usrbinperl -w

------------------------------------------------------------------------

send_ngalertpl

send alert to nagios using NSCA agent

usage

see usage below

------------------------------------------------------------------------

use strict

use warnings

use SysHostname

use GetoptLong

my $me = send_ngalertpl

my ($help $verbose)

my $nscadir = usrlocalnsca

my $host = hostname()

my ($service $rc $info $data $msg)

my statusMap = (OK =gt 0 UP =gt 0

DOWN =gt 1 WARN =gt 1

UNR =gt 2 UNREACHABLE =gt 2 CRIT =gt 2 CRITICAL =gt 2)

-------------------------------------------------------------------

process args

GetOptions (nscadir|d=s =gt $nscadir

host|n=s =gt $host

status|c=s =gt $rc

service|s=s =gt $service

statusinfo|i=s =gt $info

perfdata|p=s =gt $data

message|m=s =gt $msg

help|h =gt $help

verbose|v =gt $verbose)

my ($svrcfg $clcfg $send_nsca_bin) = ($nscadircfgnsca_servercfg

$nscadircfgnsca_clientcfg

$nscadirbinsend_nsca)

-------------------------------------------------------------------

usage

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

58

if ($help)

printf $me send alert message to Nagios server using NSCA agent

Usage

$me [-h] [-v] [-d dir] [-n host] [-s service] -c status -m message

Options

[-nscadir|d ltnsca-dirgt] dir with nsca client installed

usrlocalnsca by default

[-host|n lthostgt] host name for alert (localhost by default)

[-service|s ltservicegt service for alert

sends a service alert if provided

sends a host alert if missing

-status|c ltstatusgt alert status code

Host Status

UP DOWN UNR (unreachable)

Service Status

OK WARN CRIT (critical)

-message|m ltmessagegt message to send

-statusinfo|i ltinfogt status info to send

-perfdata|d ltdatagt performance data to send

-h help (this message)

-v verbose

Payload

ltmessagegt - [ltstatusinfogt] perfdata

Pl refer Nagios documentation for format for the payload

Sample

Send host alert

$me -n demo-tr -c OK -i Status=UP -m remote host up

Send service alert

$me -n demo-tr -s remote_alert -c CRIT -i Status=Critical -p Warnings=101010

Errors=31010 -m remote service alert

exit 1

-------------------------------------------------------------------

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

59

validate args env

die Missing message arg -m unless $msg

die Missing status-code arg -c unless $rc

die Unknown status-code $rc unless exists $statusMap$rc

die Error reading server config $svrcfg unless -e $svrcfg

die Error reading client config $svrcfg unless -e $clcfg

-------------------------------------------------------------------

process server cfg

my svrcfgMap

open (SVRCFG $svrcfg) or die Error opening server config file $svrcfg

print Reading server config $svrcfgn

while (ltSVRCFGgt)

next if

my ($tag $val) = split =

chop

chomp $val

$svrcfgMap$tag = $val

print svrcfgMap$tag = $valn if $verbose

close (SVRCFG)

my $status = $statusMap$rc

if ($verbose)

printf -- input --

nscadir $nscadir

server cfg $svrcfg

client cfg $clcfg

send_nsca $send_nsca_bin

hostname $host

service $service

status $rc ($status)

Message ---

Status Info $info

Perf Data $data

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

60

my $cmd

my $cmdtype = host

$msg = - [$info] if $info

$msg = | $data if $data

if ($service)

$cmdtype = service

$cmd = $host$service$status$msg

else

$cmd = $host$status$msg

print Command n$cmdn if $verbose

printf send_nsca command = | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg

-d n if $verbose

printf starting send_nsca client n

open (SEND_NSCA | $send_nsca_bin -H $svrcfgMaphost -p $svrcfgMapport -c $clcfg -d ) ||

die Error opening send_nsca ($send_nsca_bin)

print sending $cmdtype command $cmdn

print SEND_NSCA $cmdn

while (ltSEND_NSCAgt)

print

close (SEND_NSCA)

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

61

1043 nsca_init

binbash

nsca Startup script for the Nagios Service Check Acceptor

chkconfig - 85 15

description Nagios service check acceptor daemon

processname nsca

config usrlocalnscacfgnscacfg

pidfile varrunnscanscapid__NOT_USED

BEGIN INIT INFO

Provides nsca

Required-Start $local_fs $remote_fs $network $named

Required-Stop $local_fs $remote_fs $network

Should-Start distcache

Short-Description start and stop Nagios Service check Accaptor Server

Description Nagios extension

END INIT INFO

Start nsca in the C locale by default

NSCA_LANG=$NSCA_LANG-C

This will prevent initlog from swallowing up a pass-phrase prompt if

mod_ssl needs a pass-phrase from the user

INITLOG_ARGS=

Set NSCA=usrsbinnscaworker in etcsysconfignsca to use a server

with the thread-based worker MPM BE WARNED that some modules may not

work correctly with a thread-based MPM notably PHP will refuse to start

nsca=$NSCA-usrlocalnscasbinnsca

nscacfg=usrlocalnscacfgnscacfg

prog=nsca

pidfile=$PIDFILE-varrunnscanscapid

lockfile=$LOCKFILE-varlocksubsysnsca

RETVAL=0

STOP_TIMEOUT=$STOP_TIMEOUT-10

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

62

OPTIONS=-c $nscacfg --inetd

OPTIONS=-c $nscacfg

start()

echo -n $Starting $prog

LANG=$NSCA_LANG $nsca $OPTIONS

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp touch $lockfile

pgrep -f usrlocalnscasbinnscanscacfg$

return $RETVAL

When stopping nsca a delay (of default 10 second) is required

before SIGKILLing the nsca parent this gives enough time for the

nsca parent to SIGKILL any errant children

stop()

echo -n $Stopping $prog

pgrep -f usrlocalnscasbinnscanscacfg$ ampamp pkill -f

usrlocalnscasbinnscanscacfg$

RETVAL=$

echo

[ $RETVAL = 0 ] ampamp rm -f $lockfile $pidfile

See how we were called

case $1 in

start)

start

stop)

stop

status)

pid=$(pgrep -f usrlocalnscasbinnscanscacfg$)

if [ -z $pid ] then

echo nsca is not running

else

echo nsca (pid $pid) is running

fi

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

63

restart)

stop

start

)

echo $Usage $prog start|stop|restart|condrestart|try-restart|force-

reload|reload|status|fullstatus|graceful|help|configtest

RETVAL=2

esac

exit $RETVAL

1044 NSCA Config

==gt cfgnscacfg lt==

log_facility=daemon

pid_file=varrunnscapid

server_port=5667

nsca_user=nagios

nsca_group=nagios

debug=0

command_file=varspoolnagioscmdnagioscmd

alternate_dump_file=usrlocalnagiosvarrwnscadump

aggregate_writes=0

append_to_file=0

max_packet_age=30

decryption_method=1

==gt cfgnsca_clientcfg lt==

encryption_method=1

==gt cfgnsca_servercfg lt==

host=127001

port=5667

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

64

105 NRPE

1051 Sample server config file

Each client host to be monitored with NRPE should have a config file on the server Here is a sample config file that

should be placed on Nagios objects directory (etcnagiosobjects by default) on the Nagios server

This config file does the following checks

regular host ping check (non NRPE)

check users (NRPE)

check load (NRPE)

cat etcnagiosobjectsnrpe-test1cfg

define host

use generic-host

host_name ubuntu-svr

alias ubuntu

address 1921681108

max_check_attempts 10

define service

use generic-service

host_name ubuntu-svr

service_description PING

check_command check_ping100020500060

define service

use generic-service

host_name ubuntu-svr

service_description Current Load

check_command check_nrpecheck_load

define service

use generic-service

host_name ubuntu-svr

service_description Total Processes

check_command check_nrpecheck_users

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped

Solace Nagios Toolset Integration

65

1052 Sample Client Config

cat etcnagiosnrpecfg

log_facility=daemon

pid_file=varrunnagiosnrpepid

server_port=5666

nrpe_user=nagios

nrpe_group=nagios

allowed_hosts=127001192168562011921681223

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

command[check_users]=usrlibnagiospluginscheck_users -w 5 -c 10

command[check_load]=usrlibnagiospluginscheck_load -w 15105 -c 302520

command[check_hda1]=usrlibnagiospluginscheck_disk -w 20 -c 10 -p devhda1

command[check_zombie_procs]=usrlibnagiospluginscheck_procs -w 5 -c 10 -s Z

command[check_total_procs]=usrlibnagiospluginscheck_procs -w 150 -c 200

include=etcnagiosnrpe_localcfg

include_dir=etcnagiosnrped