[it document] · installing nagios 4 with ssl certificate of nrpe 2.16rc2. title: installing nagios...

38
[IT DOCUMENT] PROCEDURES FOR ADMINISTERING IT SERVICES JURI CALLERI VERSION 2 MARCH 2017 Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2

Upload: others

Post on 26-Jul-2020

28 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

[IT DOCUMENT] PROCEDURES FOR ADMINISTERING IT SERVICES

JURI CALLERI

VERSION 2

MARCH 2017

Installing Nagios 4 with

SSL certificate of Nrpe

2.16RC2

Page 2: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1

Author: Juri Calleri

Version: 2

TABLE OF CONTENTS

1. INSTALLING NAGIOS 4 WITH SSL CERTIFICATE OF NRPE 2.16RC2 ................................................................... 3

1.1 Introduction .................................................................................................................... 3

2. PREPARE AN UBUNTU SERVER V. 14.04 ............................................................................................. 4

2.1 update the system and setup the network ............................................................................... 4

3. NAGIOS INSTALLATION ............................................................................................................... 5

3.1 Install Nagios Core ............................................................................................................ 5

3.2 Install Nagios Plugin .......................................................................................................... 6

3.3 Install NRPE 2.16RC2 .......................................................................................................... 6

3.4 Secure NRPE .................................................................................................................... 7

4. ENABLING THE WEB SERVER ............................................................................................................. 8

4.1 Protect Apache with SSL ..................................................................................................... 8

5 CONFIGURING THE SYSTEM ............................................................................................................. 10

5.1 Configure Nagios ............................................................................................................. 10

5.2 Configure the hosts .......................................................................................................... 11

5.3 Configure the services ....................................................................................................... 13

5.4 Configure the client ......................................................................................................... 14

5.5 Configure the contacts ...................................................................................................... 15

5.6 Configuring the NRPE for the server ...................................................................................... 16

6 INSTALL THE PLUGINS AND THE NRPE ON THE CLIENTS ................................................................................. 17

6.1 Installing the Nagios plugins ............................................................................................... 17

6.2 Installing NRPE on the client ............................................................................................... 17

6.3 Configuring NRPE on the client ............................................................................................ 18

6.4 How the server sends commands to the client.......................................................................... 20

7 CREATING THE CERTIFICATE ............................................................................................................ 21

7.1 Setup the Certification Authority ......................................................................................... 21

7.2 Create the certificate for NRPE Server .................................................................................. 22

7.3 Create the certificate for the NRPE client .............................................................................. 22

8 ADDING THE GRAPHS ................................................................................................................... 23

8.1 Installing pnp4nagios ........................................................................................................ 23

8.2 Configuring the program .................................................................................................... 24

8.3 Definition for hosts and services with web popups .................................................................... 26

8.4 Understanding pnp4nagios .................................................................................................. 26

9 SENDING EMAIL NOTIFICATION WITH PHP ............................................................................................... 29

9.1 Add php email command definition ....................................................................................... 29

9.2 Apply the new email template ............................................................................................ 30

10 USEFUL PLUGINS ...................................................................................................................... 35

10.1 Top 4 plugins to download ................................................................................................ 35

11 USEFUL NAGIOS COMMANDS ........................................................................................................... 37

Page 3: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 2

Author: Juri Calleri

Version: 2

11.1 Check validity of the script ............................................................................................... 37

11.2 Start Nagios service ........................................................................................................ 37

11.3 Reload the service .......................................................................................................... 37

11.4 Stop the service ............................................................................................................. 37

11.5 Restart Nagios and xinetd ................................................................................................. 37

Page 4: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 3

Author: Juri Calleri

Version: 2

1. INSTALLING NAGIOS 4 WITH SSL CERTIFICATE OF NRPE 2.16RC2

1.1 INTRODUCTION

THIS DOCUMENT CONTAINS THE INFORMATION AND STEPS TO PROCEED TO INSTALL AND CONFIGURE THE NAGIOS MONITORING SYSTEM ON AN UBUNTU SERVER 14.04 USING THE NRPE 2.16RC2. NRPE 2.16RC2 IS STILL NOT AT ITS LAST (OR DEFINED STABLE) VERSION BUT, TESTS SHOWS THAT THIS RELEASE DOES NOT HAVE PARTICULAR ISSUE. THIS PLUGIN IS A GREAT IMPROVEMENT SINCE IT PROVIDES AN INCREASE OF SECURITY THANKS TO THE USE OF SERVER-CLIENT CERTIFICATES AND AN IMPROVEMENT OF THE ENCRYPTION SYSTEM. FOR AN EASY VIEW ON THE CONFIGURATION FILE, SOME SAMPLES OF THE SAME ARE INSIDE THIS DOCUMENT.

Page 5: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 4

Author: Juri Calleri

Version: 2

2. PREPARE AN UBUNTU SERVER V. 14.04

2.1 UPDATE THE SYSTEM AND SETUP THE NETWORK

As root, run:

apt-get update && apt-get upgrade -y

Set up a static IP:

vi /etc/network/interfaces

# The primary network interface

auto eth0

iface eth0 inet static

address 192.168.0.X

netmask 255.255.255.0

network 192.168.0.0

broadcast 192.168.0.255

gateway 192.168.0.Y

dns-nameservers 192.168.0.Z 8.8.8.8

Restart the network card to apply the new IP:

ifdown eth0

ifup eth0

Install OpenSSH server (if not installed already):

apt-get install openssh-server -y

And edit sshd_config to disable rootLogin:

vi /etc/ssh/sshd_conf

PermitRootLogin no

Restart the ssh daemon:

service ssh restart

Page 6: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 5

Author: Juri Calleri

Version: 2

3. NAGIOS INSTALLATION

3.1 INSTALL NAGIOS CORE

Install the LAMP stack and secure mysql installation, when done proceed with the creation of the user that will run Nagios and its group:

useradd NagiosUser

groupadd NagiosGroup

usermod -a -G Nagiosgroup NagiosUser

passwd NagiosUser

We will install Nagios core from source therefore we need to install these packages:

apt-get install build-essential libgd2-xpm-dev openssl libssl-dev

xinetd apache2-utils unzip –y

If you want to send the emails using postfix, also install:

apt-get install postfix mailutils -y

Move to your TEMP folder or create a download folder, and download Nagios core (edit the url to match the latest Nagios core version):

curl -L -O

https://assets.nagios.com/downloads/nagioscore/releases/nagios-

4.2.4.tar.gz

tar xvf nagios-*.tar.gz

cd nagios-*

./configure --with-nagios-user=NagiosUser --with-nagios-

group=NagiosGroup --with-command-group=NagiosGroup --with-

mail=/usr/sbin/sendmail

With --with-mail=/usr/sbin/sendmail we add the ability to Nagios to send emails using postfix. In the event you wish to use the PHP_Mailer, you may skip this and the postfix installation. A benefit of using the PHP_Mailer is the possibility to edit the body of the emails in HTML format.

If instead you prefer to keep postfix, locate the mail executable in commands.cfg and make

sure it matches your OS’s /bin/mail path. In Ubuntu is in /usr/bin/mail.

Now, compile and install Nagios:

make all

make install

make install-init

-- You can stop here if you are upgrading Nagios version --

make install-commandmode

make install-config

/usr/bin/install -c -m 644 sample-config/httpd.conf

/etc/apache2/sites-available/nagios.conf

To issue external commands to Nagios, add the web server user to Nagios group:

Page 7: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 6

Author: Juri Calleri

Version: 2

usermod -G NagiosGroup www-data

3.2 INSTALL NAGIOS PLUGIN

Create a subfolder for downloading the plugins, edit the url to match their latest version, and:

curl -L -O http://nagios-plugins.org/download/nagios-plugins-

2.1.4.tar.gz

tar xvf nagios-plugins-*.tar.gz

cd nagios-plugins-*

./configure --with-nagios-user=NagiosUser --with-nagios-

group=NagiosGroup --with-openssl

make

make install

3.3 INSTALL NRPE 2.16RC2

UPDATE: The version used below is not the latest anymore, it has been removed from the repository as well. The following code and the certificate check would probably work with the latest version of the plugin, the NRPE 3.0.1. But it hasn't been tested yet. Create a subfolder for NRPE, and:

curl -L -O https://github.com/NagiosEnterprises/nrpe/archive/nrpe-2-

16-RC2.zip

unzip nrpe-*.zip -d nrpe

cd nrpe/nrpe*

chmod +x configure

./configure --with-nrpe-user=NagiosUser --with-nrpe-

group=NagiosGroup --with-nagios-user=NagiosUser --with-nagios-

group=NagiosGroup --with-ssl=/usr/bin/openssl --with-ssl-

lib=/usr/lib/x86_64-linux-gnu

make all

make install

make install-xinetd

make install-daemon-config

Only the xinetd that run on the client must be edited to allow the Nagios server to connect:

vi /etc/xinetd.d/nrpe

only_from = IP_OF_SERVER

Restart xinetd, if you updated nrpe on the monitored server:

service xinetd restart

Page 8: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 7

Author: Juri Calleri

Version: 2

3.4 SECURE NRPE

To secure Nrpe with the certificate, the section #7 shows how to create the certificate and apply it to the configuration.

The main error messages you may get are the following:

Status Information: Error: could not use certificate file '/usr/local/nagios/etc/ssl/client_certs/nag_serv.pem'.

CHECK_NRPE: Error - Could not complete SSL handshake with 192.168.10.219: 1

A temporary solution is to set the ../ssl/ folder and certificates permissions to 777. If now the nrpe is working, change again the permissions to secure the certificates from unauthorized users. 555 is a good set of permissions to use in this case, because both the owner, group and others can only execute it.

Page 9: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 8

Author: Juri Calleri

Version: 2

4. ENABLING THE WEB SERVER

4.1 PROTECT APACHE WITH SSL

Enable auth digest (optional):

a2enmod auth_digest

htdigest -c /usr/local/nagios/etc/.digest_pw "Nagios Access"

nagiosadmin

Add the created file in nagios.conf, for both /sbin and /share and Apache <2.3 and >=2.3 sections:

vi /etc/apache2/sites-available/nagios.conf

AuthName "Nagios Access"

AuthType Digest

AuthUserFile /usr/local/nagios/etc/.digest_pw

Use Apache2 ssl, rewrite and cgi mods:

a2enmod ssl

a2enmod rewrite

a2enmod cgi

Create the certificate for the web server that lasts 10 years:

openssl genrsa -des3 -out server.3des-key 1024

openssl rsa -in server.3des-key -out server.key

openssl req -new -key server.key -x509 -out server.crt -days 3650

chmod 600 server.key

rm server.3des-key

mv server.crt /etc/ssl/

mv server.key /etc/ssl/private/

And add these certificates in default-ssl.conf:

vi /etc/apache2/sites-available/default-ssl.conf

SSLCertificateFile /etc/ssl/server.crt

SSLCertificateKeyFile /etc/ssl/private/server.key

Edit once again nagios.conf to add SSLRequireSSL in both /sbin and /share:

vi /etc/apache2/sites-available/nagios.conf

<Directory "/usr/local/nagios/sbin">

SSLRequireSSL

[...]

<Directory "/usr/local/nagios/share">

SSLRequireSSL

Apply the new SSL and restart Apache2:

Page 10: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 9

Author: Juri Calleri

Version: 2

a2ensite default-ssl.conf

service apache2 restart

Create a symbolic link to enable the new site:

ln -s /etc/apache2/sites-available/nagios.conf /etc/apache2/sites-

enabled/

And enable Nagios to start on boot:

ln -s /etc/init.d/nagios /etc/rcS.d/S99nagios

Now you can access its the Nagios web interface: https://nagios_server/nagios username: nagiosadmin password: *The one you choose earlier*

Page 11: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 10

Author: Juri Calleri

Version: 2

5 CONFIGURING THE SYSTEM

5.1 CONFIGURE NAGIOS

The main configuration file for Nagios core is nagios.cfg in /usr/local/nagios/etc. This file, among other settings, allows the addition of other paths for smaller configuration file. A clean way to configure Nagios is to understand its operation and build the configuration settings separately. This chapter shows how to split and manage the configurations. First make a backup of nagios.cfg:

cd /usr/local/nagios/etc

cp nagios.cfg nagios.cfg.original

With in mind that rows that start with the sharp sign ( # ) are comments, delete accordingly this sign from nagios.cfg where you want to activate its setting, just like below:

vi /usr/local/nagios/etc/nagios.cfg

#cfg_dir=/usr/local/nagios/etc/servers

[ and delete the ‘#’ to activate the ‘servers’ folder ]

[ Locate then this row, which is above the previous ]

# You can specify individual object config files as shown below:

[ And add as many configuration file as needed ]

cfg_file=/usr/local/nagios/etc/objects/commands.cfg

cfg_file=/usr/local/nagios/etc/objects/contacts.cfg

cfg_file=/usr/local/nagios/etc/objects/timeperiods.cfg

cfg_file=/usr/local/nagios/etc/objects/nrpecommands.cfg

cfg_file=/usr/local/nagios/etc/objects/services.cfg

cfg_file=/usr/local/nagios/etc/objects/hostconf.cfg

cfg_file=/usr/local/nagios/etc/objects/mailtemplate.cfg

[ Or, if you are confident enough you may use a single cfg_dir,

just like for ‘servers’, and place all of the above cfg_file

inside your cfg_dir ]

cfg_dir=/usr/local/nagios/etc/servers

#cfg_dir=/usr/local/nagios/etc/printers

#cfg_dir=/usr/local/nagios/etc/switches

#cfg_dir=/usr/local/nagios/etc/routers

cfg_dir=/usr/local/nagios/etc/myconfiguration

Basically, one can choose if adding the single configuration file or a whole folder. This document uses both.

A folder with all of the servers: cfg_dir=/usr/local/nagios/etc/servers

and the single setting: cfg_file=/usr/local/nagios/etc/objects/nrpecommands.cfg

nrpecommands.cfg is the heart of the nrpe plugin, we can proceed to create this file:

Page 12: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 11

Author: Juri Calleri

Version: 2

vi /usr/local/nagios/etc/objects/nrpecommands.cfg

define command{

command_name check_nrpe

command_line $USER1$/check_nrpe -H $HOSTADDRESS$ -c $ARG1$

}

define command{

command_name check_nrpe_cert

command_line $USER1$/check_nrpe -H $HOSTADDRESS$ -C

/usr/local/nagios/etc/ssl/server_certs/nag_serv.pem -K

/usr/local/nagios/etc/ssl/server_certs/nag_serv.key -A

/usr/local/nagios/etc/ssl/ca/ca_cert.pem -c $ARG1$ -t 15

}

As you can see, there are both the versions of the nrpe command, this document though will

focus only in having the nrpe working with the certificate: check_nrpe_cert.

We now need to edit the contacts.cfg and enter the email address you are going to use to receive Nagios notification:

vi /usr/local/nagios/etc/objects/contacts.cfg

email nagios@localhost ; <<***** CHANGE THIS TO YOUR EMAIL ADDRESS

******

[ You may edit the name too, the most important is the email ]

5.2 CONFIGURE THE HOSTS

Each host is created grabbing information from the other configuration files. These information are the services, the timetable, the group of the servers the host belongs to [...] All of these information are called “definitions”. A configuration file is a mix of definitions, the exact same definitions used inside the “templates.cfg”. But, templates.cfg is generic and contains all the definitions used for the hosts, rather, we wish each file for each configuration, this way it is more clear to work with.

A suggestion is to open your templates.cfg and check its content. Then, we can create the single files:

vi /usr/local/nagios/etc/objects/hostconf.cfg

# Suppose you have Linux and Windows servers, or production and test

# server, or simply using groups to differentiate one host from another

## HOST GROUP DEFINITION ##

define hostgroup{

hostgroup_name Test Hostgroup

alias Test Servers

}

# This is the hostgroup, used show together all the servers with the

# same scope, can be test, webserver, domain controllers and so on..

Page 13: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 12

Author: Juri Calleri

Version: 2

## HOST BASE DEFINITION ##

define host{

Name linux-host

notifications_enabled 1

event_handler_enabled 1

flap_detection_enabled 1

process_perf_data 1

retain_status_information 1

retain_nonstatus_information 1

notification_period 24x7

check_period 24x7

retry_interval 1

max_check_attempts 5

check_command check-host-alive

contact_groups avengers

register 0

}

# This is a template that can match a very base server.

# All the servers with same scope may use the same base definition

# so they become members of the linux-host group.

# Every setting applied here is applied for all of its members.

## HOST DEFINITION ##

define host{

Name linux-test

Use linux-host

Hostgroups Test Hostgroup

check_interval 2

notification_period workhours

notification_interval 30

notification_options d,u,r

register 0

}

# This host is using the base template but it is still possible

# to override the settings, just like happened with notification_period

# and the lasts are applied

In all of them appears the register 0, which means that this is not a real host but a template.

We will configure the hosts inside /usr/local/nagios/etc/servers.

Page 14: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 13

Author: Juri Calleri

Version: 2

5.3 CONFIGURE THE SERVICES

Same as the host, the services may use groups but it is more useful to create the templates for the services:

vi /usr/local/nagios/etc/objects/services.cfg

## SERVICES DEFINITIONS ##

define service{

Name generic-service

active_checks_enabled 1

passive_checks_enabled 0

parallelize_check 1

obsess_over_service 0

check_freshness 0

notifications_enabled 1

event_handler_enabled 1

flap_detection_enabled 1

process_perf_data 0

retain_status_information 1

retain_nonstatus_information 1

is_volatile 0

check_period 24x7

max_check_attempts 3

check_interval 5

retry_interval 2

contact_groups avengers

notification_options w,u,c,r

notification_interval 60

notification_period 24x7

register 0

}

# This is the service template for a generic service

# it works exactly like the host template.

# And now we have a different kind of service!

# Below is the definition of a service to be used with a plugin able

# to check the log files.

# The application that creates this log files, is fail2ban.

# I created this service for Nagios with the purpose to define a

# different kind of service check

define service{

Name fail2banlog-service

Use generic-service

flap_detection_enabled 0

retain_status_information 0

retain_nonstatus_information 0

is_volatile 1

Page 15: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 14

Author: Juri Calleri

Version: 2

max_check_attempts 1

check_interval 4

retry_interval 2

notification_interval 0

register 0

}

# Using the base template for services, this template does not save

# the data of the service (is volatile 1), also does not check 3 times

# before sending the alarm (max_check_attempts 1 ), and can pass

# between OK and ALARM status without being blocked by Nagios for being

# “flapping” (flap_detection_enabled 0).

This document wants to be just an overview on Nagios and its configuration, which leaves the reader to read the Nagios manual and understand every options that this document does not cover.

5.4 CONFIGURE THE CLIENT

Each .cfg file is a host, we can name it after the real server hostname simply to differentiate this from another:

vi /usr/local/nagios/etc/servers/fileserver-test.cfg

## HOST ##

define host{

use linux-test

host_name fileserver-test

alias Fileserver test

address 192.168.10.11

}

define service{

name fileserver-test-service

host_name fileserver-test

register 0

}

## SERVICES ##

define service{

use generic-service,fileserver-test-service

service_description Check memory

check_command check_nrpe_cert!check_mem

}

define service{

use fail2banlog-service,fileserver-test-service

service_description Fail2ban

check_command check_nrpe_cert!check_log_fail2ban

}

Page 16: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 15

Author: Juri Calleri

Version: 2

This host is using the generic-service template for the check_mem command, and the

fail2banlog_service to check the logs of this program.

Also, each service needs the host_name definition to perform their check on the correct host. Using a service definition template you can apply to the service the corresponding host.

service description is the name appearing on the Nagios web panel.

host_name MUST match the one used under the ## HOST ##.

check_command calls the command check_mem using the server’s command

check_nrpe_cert.

check_mem is defined in the NRPE.cfg file on the host machine.

5.5 CONFIGURE THE CONTACTS

You may have noticed that in the previous chapters, “avengers” was used in the

contact_groups.

A simple contacts.cfg with two users is as follows:

vi /usr/local/nagios/etc/objects/contacts.cfg

## CONTACTS DEFINITION ##

define contact{

name generic-contact

service_notification_period 24x7

host_notification_period 24x7

service_notification_options w,u,c,r,f,s

host_notification_options d,u,r,f,s

service_notification_commands notify-service-by-email

host_notification_commands notify-host-by-email

register 0

}

## CONTACTS ##

define contact{

contact_name Tony

use generic-contact

alias Iron-man

email [email protected]

}

define contact{

contact_name Steve

use generic-contact

alias Captain America

email [email protected]

}

## CONTACT GROUPS ##

define contactgroup{

contactgroup_name avengers

alias Nagios Administrators

Page 17: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 16

Author: Juri Calleri

Version: 2

members Tony, Steve

}

Now you know why!

All the .cfg files follows the same logic. If we want to use contacts for office hours and for night shifts all we have to do is to add a new contact definition, set it up and register 0. The: service_notification_commands notify-service-by-email

host_notification_commands notify-host-by-email

are both defined inside commands.cfg. For local commands, those used on the Nagios monitoring server itself, there is no need to use the certificate.

5.6 CONFIGURING THE NRPE FOR THE SERVER

NRPE config file in /usr/local/nagios/etc/nrpe.cfg is installed at the moment of the NRPE plugin installation.

If it should act as a server or client depends by its configuration.

The server configuration can be reduced to a few lines only, there is no need of leaving the NRPE user because the connection is managed by xinetd. Several other settings were removed from this configuration. Do not forget to make a backup of the original file:

cd /usr/local/nagios/etc

cp nrpe.cfg nrpe.cfg.original

nrpe.cfg contains useful information, before using the configuration below it is better to have a look at it and once understood it you can erase its content and update:

>nrpe.cfg

vi nrpe.cfg

log_facility=daemon

pid_file=/var/run/nrpe.pid

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

ssl_version=TLSv1.2+

ssl_use_adh=1

ssl_cipher_list=ALL:!MD5:@STRENGTH

ssl_client_certs=2

ssl_logging=0xff

Logs are enabled and set to full-log. When your configuration is working the ssl_logging directive can be turned off.

Page 18: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 17

Author: Juri Calleri

Version: 2

6 INSTALL THE PLUGINS AND THE NRPE ON THE CLIENTS

6.1 INSTALLING THE NAGIOS PLUGINS

The Nagios plugin installation for the hosts follows the same commands used in the previous chapter.

Follow the steps to set-up the network with a static IP

Create the user and group for Nagios, just like earlier in the document:

useradd NagiosUser

groupadd NagiosGroup

usermod -a -G NagiosGroup NagiosUser

passwd NagiosUser

Run the repository update command on the host and install a few packages:

apt-get update

apt-get install build-essential openssl libssl-

dev xinetd unzip libsys-statistics-linux-perl -y

libsys-statistics-linux-perl is used by the plugin

check_linux_stats.pl on the Nagios plugins hub. This plugin is good for checks on cpu, memory, disk and many other.

You can download it from Nagios Exchange and chmod +x to make it executable

when copied to /usr/local/nagios/libexec. But, if your plans are to use graphs for the CPU, probably you wish to use another plugin called check_cpu.py, because this plugin shows the load on the CPU and not the idle percentage.

Also, check_cpu.py comes with a template for pnp4nagios. More about pnp4nagios at its section.

Now use the same command used for installing the plugin on the Nagios server:

curl -L -O http://nagios-plugins.org/download/nagios-

plugins-2.1.4.tar.gz

tar xvf nagios-plugins-*.tar.gz

cd nagios-plugins-*

./configure --with-nagios-user=NagiosUser --with-nagios-

group=NagiosGroup --with-openssl

make

make install

6.2 INSTALLING NRPE ON THE CLIENT

Also the NRPE installation is the same as the one for the server.

Follow these commands:

curl -L -O

https://github.com/NagiosEnterprises/nrpe/archive/nrpe-2-

16-RC2.zip

Page 19: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 18

Author: Juri Calleri

Version: 2

unzip nrpe-*.zip -d nrpe

cd nrpe/nrpe*

chmod +x configure

./configure --with-nrpe-user=NagiosUser --with-nrpe-

group=NagiosGroup --with-nagios-user=NagiosUser --with-

nagios-group=NagiosGroup --with-ssl=/usr/bin/openssl --

with-ssl-lib=/usr/lib/x86_64-linux-gnu

make all

make install

make install-xinetd

make install-daemon-config

And remember to update xinetd:

vi /etc/xinetd.d/nrpe

only_from = IP_OF_NAGIOS_SERVER

When done, restart it:

service xinetd restart

6.3 CONFIGURING NRPE ON THE CLIENT

All we have to do now is to set up the certificate and update the NRPE conf file.

This chapter is focusing on creating the NRPE config file for the client and the certificate

creation is left for the next section.

Previously we created the nrpecommands.cfg to store the nrpe command and certificates.

Now, the certificates are defined in the client’s nrpe config file.

Again, make a copy of the file and then erase it to copy this new settings:

cd /usr/local/Nagios/etc

cp nrpe.cfg nrpe.cfg.original

>nrpe.cfg

vi nrpe.cfg

log_facility=daemon

pid_file=/var/run/nrpe.pid

dont_blame_nrpe=0

allow_bash_command_substitution=0

debug=0

command_timeout=60

connection_timeout=300

ssl_version=TLSv1.2+

ssl_use_adh=1

ssl_cipher_list=ALL:!MD5:@STRENGTH

ssl_cacert_file=/usr/local/nagios/etc/ssl/ca_cert.pem

ssl_cert_file=/usr/local/nagios/etc/ssl/monitored.pem

ssl_privatekey_file=/usr/local/nagios/etc/ssl/monitored.key

ssl_client_certs=2

Page 20: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 19

Author: Juri Calleri

Version: 2

ssl_logging=0xff

command[check_mem]=/usr/local/nagios/libexec/check_linux_stats.pl

-M -w 100,25 -c 100,50

command[check_log_fail2ban]=/usr/local/nagios/libexec/check_log3.pl -l

/var/log/fail2ban.log -p 'Ban' -w 1 -c 5

No nrpe user needed here as well, because xinetd is taking care of the connection.

You can see it differs from the server’s one because this contains both the certificate and the

commands that the server asks the client to run:

command[check_mem]=/usr/local/nagios/libexec/check_linux_stats.pl -M -w

100,25 -c 100,50

command[check_log_fail2ban]=/usr/local/nagios/libexec/check_log3.pl -l

/var/log/fail2ban.log -p 'Ban' -w 1 -c 5

Page 21: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 20

Author: Juri Calleri

Version: 2

6.4 HOW THE SERVER SENDS COMMANDS TO THE CLIENT

The server contain this definition in the client’s configuration inside the ../servers folder:

define service{

use generic-service,fileserver-test-service

service_description Check memory

check_command check_nrpe_cert!check_mem

}

And the client have the check_mem defined in the nrpe.cfg file.

command[check_mem]=/usr/local/nagios/libexec/check_linux_stats

The parameters after the plugin are used to send to the client what service to check and what it

is reported as good, warning or critical status.

On the client:

cd /usr/local/nagios/libexec

./check_linux_stats.pl -M -w 100,25 -c 100,50

if you would run this command, the reply is the same that will be sent to Nagios, where in the

case of this plugin: -M is for “memory”, -w is the warning and –c the critical.

Nagios uses Linux exit codes to understand the output of the commands:

exit 0 – Good

exit 1 – Warning

exit 2 – Critical

exit 3 – Unknown # Usually due to communication errors between the server

and client

This means that it is easy to create own plugins and let Nagios run them.

Among what is reported to Nagios, there are the performance_data, more on this in the

pnp4nagios section.

performance_data are detailed information about the last check, these are widely used when

creating graphs but are flexible too.

Page 22: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 21

Author: Juri Calleri

Version: 2

7 CREATING THE CERTIFICATE

7.1 SETUP THE CERTIFICATION AUTHORITY

The certificate proposed here have a validity of 10 years, just like the one for apache2.

Nagios server will become a Certification Authority, obviously it is not trusted on the internet

and its certificates are only valid for internal use, which is enough for us.

First, set up the directories:

mkdir -p -m 750 /usr/local/nagios/etc/ssl

chown root.NagiosGroup /usr/local/nagios/etc/ssl

cd /usr/local/nagios/etc/ssl

mkdir -m 750 ca

chown root.root ca

mkdir -m 750 server_certs

chown root.NagiosGroup server_certs

mkdir -m 750 client_certs

chown root.NagiosGroup client_certs

And set up the Certification Authority (CA):

cd /usr/local/nagios/etc/ssl

mkdir –p –m 700 demoCA/newcerts

touch demoCA/index.txt

echo "01" > demoCA/serial

chown -R root.root demoCA

chmod 600 demoCA/serial

chmod 600 demoCA/index.txt

chmod 700 demoCA

Then, create the CA:

cd /usr/local/nagios/etc/ssl/ca

openssl req -x509 -newkey rsa:4096 -keyout ca_key.pem -out

ca_cert.pem -utf8 -days 3650

Page 23: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 22

Author: Juri Calleri

Version: 2

7.2 CREATE THE CERTIFICATE FOR NRPE SERVER

Create the NRPE Server Certificate Request (CSR, Certificate Signing Request) and sign it:

cd /usr/local/nagios/etc/ssl/server_certs

openssl req -new -newkey rsa:2048 -keyout nag_serv.key -out

nag_serv.csr -nodes

cd /usr/local/nagios/etc/ssl

openssl ca -extensions usr_cert -days 3650 -notext -md sha256 -

keyfile ca/ca_key.pem -cert ca/ca_cert.pem -in

server_certs/nag_serv.csr -out server_certs/nag_serv.pem

chown root.NagiosGroup server_certs/nag_serv.pem

chmod 440 server_certs/nag_serv.pem

Copy nag-serv.pem, nag-serv.key and ca/ca_cert.pem files to the Nagios’s ssl folder.

The paths will match those used in the nrpecommands.cfg.

But, if your CA is, just like in this documentation, on your Nagios server, there is no need to

move the certificate anywhere. The paths points already to this location.

7.3 CREATE THE CERTIFICATE FOR THE NRPE CLIENT

Create the CSR of the NRPE Client and sign it:

cd /usr/local/nagios/etc/ssl/server_certs

openssl req -new -newkey rsa:2048 -keyout monitored.key -out

monitored.csr -nodes

Follow the prompts. The -nodes at the end of the lines tells openssl to generate the key

without a passphrase. Leave it off, if you want someone to enter a passphrase whenever the

machine boots.

IMPORTANT: Do not create certificates using the same Common Name (CN).

You can use one certificate for all of the servers to monitor (less secure, but acceptable).

Now you need to sign the CSR with the CA key:

cd /usr/local/nagios/etc/ssl

openssl ca -days 3650 -notext -md sha256 -keyfile ca/ca_key.pem -cert

ca/ca_cert.pem -in client_certs/monitored.csr -out

client_certs/monitored.pem

chown root.NagiosGroup client_certs/monitored.pem

chmod 444 client_certs/monitored.pem

Copy monitored.pem, monitored.key and ca/ca_cert.pem files to every machine in the

Nagios’s ssl folder.

The paths will match those used in the nrpe.cfg.

Page 24: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 23

Author: Juri Calleri

Version: 2

8 ADDING THE GRAPHS

8.1 INSTALLING PNP4NAGIOS

Change directory to your download folder, and enter:

apt-get install rrdtool librrds-perl

curl -L -O http://docs.pnp4nagios.org/_media/dwnld/pnp4nagios-

head.tar.gz

tar -xvzf pnp4nagios*

cd pnp4nagios

./configure --with-nagios-user=NagiosUser --with-nagios-

group=NagiosGroup --with-httpd-conf=/etc/apache2/sites-available

Follow the prompts. The output is similar to:

*** Configuration summary for pnp4nagios-0.6.2 23-12-2009 ***

General Options:

------------------------- -------------------

Nagios user/group: NagiosUser NagiosGroup

Install directory: /usr/local/pnp4nagios

HTML Dir: /usr/local/pnp4nagios/share

Config Dir: /usr/local/pnp4nagios/etc

Location of rrdtool binary: /usr/bin/rrdtool Version 1.2.12

RRDs Perl Modules: FOUND (Version 1.2012)

RRD Files stored in: /usr/local/pnp4nagios/var/perfdata

process_perfdata.pl Logfile:

/usr/local/pnp4nagios/var/perfdata.log

Perfdata files (NPCD) stored in: /usr/local/pnp4nagios/var/spool

Web Interface Options: ------------------------- ------------

-------

HTML URL: http://localhost/pnp4nagios/

Apache Config File: /etc/apache2/sites-available/pnp4nagios.conf

Review the options above for accuracy. If they look okay,

type 'make all' to compile.

Make sure it uses the same user and group defined for Nagios and that the apache config is the

right one for your system, on Ubuntu 14.04 this is correct because of the command

--with-httpd-conf=/etc/apache2/sites-available.

Compile the program:

make all

make install

Page 25: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 24

Author: Juri Calleri

Version: 2

make install-webconf

make install-config

make install-init

[ Or, all of these commands reassumed in ]

make fullinstall

If you changed the nagiosadmin webuser authentication to use the digest, do:

vi /etc/apache2/sites-available/pnp4nagios.conf

AuthName "Nagios Access"

AuthType Digest

AuthUserFile /usr/local/nagios/etc/.digest_pw

Replace this on all of it sections.

8.2 CONFIGURING THE PROGRAM

Pnp4nagios can be configured to use several modes, but the Synchronous mode will not

work.

This document is focusing to use the Bulk mode.

Check the other configurations here: Quick-start.

To use the graphs we need to have performance data enabled.

This can be done both from the Nagios config file or from the single service configuration, which

can be used to selectively enable when it is disabled on the main Nagios configuration.

To enable it globally on the Nagios config file:

vi /usr/local/nagios/etc/nagios.cfg

[ Find ]

process_performance_data=0

[ And change it to ]

process_performance_data=1

[ Then, add the code below at the end of the file ]

#

# service performance data

#

service_perfdata_file=/usr/local/pnp4nagios/var/service-perfdata

service_perfdata_file_template=DATATYPE::SERVICEPERFDATA\tTIMET::$TIMET

$\tHOSTNAME::$HOSTNAME$\tSERVICEDESC::$SERVICEDESC$\tSERVICEPERFDATA::$

SERVICEPERFDATA$\tSERVICECHECKCOMMAND::$SERVICECHECKCOMMAND$\tHOSTSTATE

::$HOSTSTATE$\tHOSTSTATETYPE::$HOSTSTATETYPE$\tSERVICESTATE::$SERVICEST

ATE$\tSERVICESTATETYPE::$SERVICESTATETYPE$

service_perfdata_file_mode=a

service_perfdata_file_processing_interval=15

service_perfdata_file_processing_command=process-service-perfdata-file

Page 26: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 25

Author: Juri Calleri

Version: 2

#

# host performance data starting with Nagios 3.0

#

host_perfdata_file=/usr/local/pnp4nagios/var/host-perfdata

host_perfdata_file_template=DATATYPE::HOSTPERFDATA\tTIMET::$TIMET$\tHOS

TNAME::$HOSTNAME$\tHOSTPERFDATA::$HOSTPERFDATA$\tHOSTCHECKCOMMAND::$HOS

TCHECKCOMMAND$\tHOSTSTATE::$HOSTSTATE$\tHOSTSTATETYPE::$HOSTSTATETYPE$

host_perfdata_file_mode=a

host_perfdata_file_processing_interval=15

host_perfdata_file_processing_command=process-host-perfdata-file

Now we need to create the command for Nagios:

vi /usr/local/nagios/etc/objects/commands.cfg

define command{

command_name process-service-perfdata-file

command_line

/usr/local/pnp4nagios/libexec/process_perfdata.pl --

bulk=/usr/local/pnp4nagios/var/service-perfdata

}

define command{

command_name process-host-perfdata-file

command_line

/usr/local/pnp4nagios/libexec/process_perfdata.pl --

bulk=/usr/local/pnp4nagios/var/host-perfdata

}

To check if the configuration is correct, pnp4nagios provides a tool in perl you can download

here.

If you receive the message PHP magic_quotes_gpc is deprecated then locate your

php.ini and set the value to Off.

Additionally, you may want to open the graphs on the main window of Nagios:

vi +325 /usr/local/nagios/etc/cgi.cfg

[ replace ]

action_url_target=_blank

[ with ]

action_url_target=main

Page 27: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 26

Author: Juri Calleri

Version: 2

8.3 DEFINITION FOR HOSTS AND SERVICES WITH WEB POPUPS

To continue the pnp4nagios configuration, another step is to add the new configuration on the

hosts and on the services, this commands shows a web popup of the graph.

On the hosts:

vi /usr/local/nagios/etc/objects/hostconf.cfg

define host{

name host-pnp

action_url

/pnp4nagios/index.php/graph?host=$HOSTNAME$&srv=_HOST_'

class='tips'

rel='/pnp4nagios/index.php/popup?host=$HOSTNAME$&srv=_HOST_

register 0

}

On the services:

vi /usr/local/nagios/etc/objects/services.cfg

define service{

name srv-pnp

action_url

/pnp4nagios/index.php/graph?host=$HOSTNAME$&srv=$SERVICEDESC$'

class='tips'

rel='/pnp4nagios/index.php/popup?host=$HOSTNAME$&srv=$SERVICEDES

C$

register 0

}

8.4 UNDERSTANDING PNP4NAGIOS

Or better, pnp4nagios must understand what is the command used for the checks:

vi /usr/local/pnp4nagios/etc/check_command/check_nrpe_cert.cfg

#

# Adapt pnp4nagios to the check_nrpe_cert command, with this template

# Command:

# check_nrpe_cert!check_mem

# ________0______| |

# ________1_______________|

#

# User ARG1

CUSTOM_TEMPLATE = 0,1

Page 28: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 27

Author: Juri Calleri

Version: 2

Pnp4nagios uses templates to show graphs based on the command executed.

Using the nrpe, the command that the server will send to the client will always be

check_nrpe_cert_check_mem, for example.

In the pnp4nagios/share/templates folder then, we need to create a php file called

check_nrpe_cert_check_mem.php and this template will match this command.

The more commands you want to show as a graph, the more templates with the command’s

name you need to create.

There are many templates you can find on internet, all of them are php file.

There are special templates too.

The pnp4nagios/share/templates.special folder, contains the templates used to show

the graph of the same service on several different servers (like the one for the PING).

An example of special template for PING, is:

vi /usr/local/nagios/pnp4nagios/share/templates.special/ping-all.php

<?php

$this->MACRO['TITLE'] = "Ping";

$this->MACRO['COMMENT'] = "For All Servers";

$services = $this->tplGetServices("","PING");

# The Datasource Name for Graph 0

$ds_name[0] = "Ping";

$opt[0] = "--title \"Ping\"";

$def[0] = "";

# Iterate through the list of hosts

$i=0;

foreach($services as $key=>$val){

if($i == 1) {$i = 2;}

$data = $this->tplGetData($val['host'],$val['service']);

#throw new Kohana_exception(print_r($a,TRUE));

$hostname = rrd::cut($data['MACRO']['HOSTNAME']);

$def[0] .= rrd::def("var$key" , $data['DS'][0]['RRDFILE'],

$data['DS'][0]['DS'] );

$def[0] .= rrd::line1("var$key", rrd::color($i),

$hostname);

$def[0] .= rrd::gprint("var$key", array("MAX", "AVERAGE"));

$i++;

}

?>

For the special templates, you don’t need to rename the file after the command name, ping-

all.php is fine.

Page 29: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 28

Author: Juri Calleri

Version: 2

The name of the service, you can change this to match your service name.

This is where you’d want to write the name of the host, use “” for all the hosts.

Like the service name, but this is the one configured in Nagios. Be careful, if what you write

here does not match any service, it will simply not work.

In order to assign a different colour to all the hosts, the $key was replaced with a different

variable, like $i.

This, because the $key will sooner or later set itself as „1“ and that is the colour of the Yellow.

The yellow is hard to see on a white background.

To avoid this, in case $i = 1 it will be set to $i = 2.

Page 30: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 29

Author: Juri Calleri

Version: 2

9 SENDING EMAIL NOTIFICATION WITH PHP

9.1 ADD PHP EMAIL COMMAND DEFINITION

It is know that the notification sent by Nagios through email are not eye-catching. There is a

very little if not none personalization of the email’s body.

Luckily, one can apply its own style to the emails thanks to the use of PHP and HTML, of course.

Create a new configuration file for the emails:

vi /usr/local/nagios/etc/objects/mailtemplate.cfg

define command{

command_name notify-service-by-email

command_line

/usr/local/nagios/libexec/nagios_service_mail

"$NOTIFICATIONTYPE$" "$HOSTNAME$" "$HOSTALIAS$" "$HOSTSTATE$"

"$HOSTADDRESS$" "$SERVICEOUTPUT$" "$SHORTDATETIME$"

"$SERVICEDESC$" "$SERVICESTATE$" "$CONTACTEMAIL$"

"$SERVICEDURATIONSEC$" "$SERVICEEXECUTIONTIME$"

"$TOTALSERVICESWARNING$" "$TOTALSERVICESCRITICAL$"

"$TOTALSERVICESUNKNOWN$" "$LASTSERVICEOK$"

"$LASTSERVICEWARNING$" "$SERVICENOTIFICATIONNUMBER$"

"$SERVICEPERFDATA$"

}

define command{

command_name notify-host-by-email

command_line /usr/local/nagios/libexec/nagios_host_mail

"$NOTIFICATIONTYPE$" "$HOSTNAME$" "$HOSTALIAS$" "$HOSTSTATE$"

"$HOSTADDRESS$" "$HOSTOUTPUT$" "$SHORTDATETIME$" "$SERVICEDESC$"

"$SERVICESTATE$" "$CONTACTEMAIL$" "$TOTALHOSTSUP$"

"$TOTALHOSTSDOWN$"

}

Remember to update the Nagios configuration file to add this new file.

And notice the name used for the definition. Same as the one used for Nagios.

Which means we need to disable those currently in use:

vi /usr/local/nagios/etc/objects/commands.cfg

# 'notify-host-by-email' command definition

#define command{

# command_name notify-host-by-email

# command_line /usr/bin/printf "%b" "***** Nagios

*****\n\nNotification Type: $NOTIFICATIONTYPE$\nHost:

$HOSTNAME$\nState: $HOSTSTATE$\nAddress: $HOSTADDRESS$\nInfo:

$HOSTOUTPUT$\n\nDate/Time: $LONGDATETIME$\n" | mail -r $ADMINEMAIL$ -s

"** $NOTIFICATIONTYPE$ Host Alert: $HOSTNAME$ is $HOSTSTATE$ **"

$CONTACTEMAIL$

# }

Page 31: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 30

Author: Juri Calleri

Version: 2

# 'notify-service-by-email' command definition

#define command{

# command_name notify-service-by-email

# command_line /usr/bin/printf "%b" "***** Nagios

*****\n\nNotification Type: $NOTIFICATIONTYPE$\n\nService:

$SERVICEDESC$\nHost: $HOSTALIAS$\nAddress: $HOSTADDRESS$\nState:

$SERVICESTATE$\n\nDate/Time: $LONGDATETIME$\n\nAdditional

Info:\n$SERVICEOUTPUT$\n\nExtra Data:\n$SERVICEPERFDATA$\n\n" | mail -r

$ADMINEMAIL$ -s "** $NOTIFICATIONTYPE$ Service Alert:

$HOSTALIAS$/$SERVICEDESC$ is $SERVICESTATE$ **" $CONTACTEMAIL$

# }

To apply comments at the beginning of every row about:

notify-host-by-email

notify-service-by-email

9.2 APPLY THE NEW EMAIL TEMPLATE

A sample of nagios_host_email is as follows:

vi /usr/local/nagios/libexec/nagios_host_email

#!/usr/bin/env php

<?php

array_shift($argv);

$f_notify_type =array_shift($argv);

$f_host_name =array_shift($argv);

$f_host_alias =array_shift($argv);

$f_host_state =array_shift($argv);

$f_host_address =array_shift($argv);

$f_host_output =array_shift($argv);

$f_long_date =array_shift($argv);

$f_serv_desc =array_shift($argv);

$f_serv_state =array_shift($argv);

$f_to =array_shift($argv);

$f_totalup =array_shift($argv);

$f_totaldown=array_shift($argv);

$subject = "$f_notify_type Host: $f_host_name";

$from ="monitoring@localhost";

$body = "<html><body><b>Notification: </b> <font

color=#CC0000>$f_notify_type</font><br/> \r\n";

$body .= "<b>Host: </b> <font

color=#007700>$f_host_alias</font> </br> \r\n";

Page 32: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 31

Author: Juri Calleri

Version: 2

$body .= "<b>Address: </b> <font

color=#005555>$f_host_address</font><br/> \r\n";

$body .= "<b>Date/Time: </b><font

color=#005555>$f_long_date</font><br/> \r\n";

$body .= "<b>More Info: </b><a

href='https://149.202.109.84/nagios/'>Nagios home page</a><br/>

\r\n";

$body .= "<b>Additional Info: </b>$f_host_output<br/> \r\n";

$body .= "<b>Total Servers Up: </b>$f_totalup<br/>";

$body .= "<b>Total Servers Down: </b>$f_totaldown<br/> \r\n";

$body .= "</body></html> \r\n";

$headers = "From: $from\r\n";

$headers = $headers."Content-type: text/html\r\n";

/* Send eMail Now... */

mail($f_to, $subject, $body, $headers);

?>

Where $from ="monitoring@localhost"; can be edited to match another user and/or

domain.

And, an example of nagios_service_mail is:

vi /usr/local/nagios/libexec/nagios_service_email

#!/usr/bin/env php

<?php

# Replace the domain "kdog.cmsnagios.com" with your

# Own Nagios Server Domain... If you have a /cms/ server

# like simplecms, replace the img src path accordingly.

array_shift($argv);

$f_notify_type =array_shift($argv); /*1*/

$f_host_name =array_shift($argv); /*2*/

$f_host_alias =array_shift($argv); /*3*/

$f_host_state =array_shift($argv); /*4*/

$f_host_address =array_shift($argv); /*5*/

$f_serv_output =array_shift($argv); /*6*/

$f_long_date =array_shift($argv); /*7*/

$f_serv_desc =array_shift($argv); /*8*/

$f_serv_state =array_shift($argv); /*9*/

$f_to =array_shift($argv); /*10*/

$f_duration = round((array_shift($argv))/60,2); /*11*/

$f_exectime =array_shift($argv); /*12*/

$f_totwarnings =array_shift($argv); /*13*/

Page 33: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 32

Author: Juri Calleri

Version: 2

$f_totcritical =array_shift($argv); /*14*/

$f_totunknowns =array_shift($argv); /*15*/

$f_lastserviceok = array_shift($argv); /*16*/

$f_lastwarning = array_shift($argv); /*17*/

$f_attempts= array_shift($argv); /*18*/

$extra_info= array_shift($argv); /*19*/

$f_downwarn = $f_duration;

$f_color="#dddddd";

if($f_serv_state=="WARNING") {$f_color="#f48400";}

if($f_serv_state=="CRITICAL") {$f_color="#f40000";}

if($f_serv_state=="OK") {$f_color="#00b71a";}

if($f_serv_state=="UNKNOWN") {$f_color="#cc00de";}

// Check If File Exists ###########

if($f_notify_type=="PROBLEM")

{

$currenttime = time();

$file_name = "/tmp/$f_host_name.$f_serv_desc.txt";

if ($f_attempts==1)

{

if(file_exists($file_name)==true) {unlink($file_name);}

$currenttime = $currenttime+round(($f_duration * 60),0);

file_put_contents($file_name, "$currenttime");

}

}

if($f_notify_type=="RECOVERY")

{

$currenttime = time();

$oldtime = time();

$file_name = "/tmp/$f_host_name.$f_serv_desc.txt";

if (file_exists($file_name)==true)

{

$oldtime = intval(file_get_contents($file_name));

}

$f_downwarn = round(($currenttime - $oldtime)/60,2);

}

$f_serv_output = str_replace("(","/",$f_serv_output);

$f_serv_output = str_replace(")","/",$f_serv_output);

$f_serv_output = str_replace("[","/",$f_serv_output);

$f_serv_output = str_replace("]","/",$f_serv_output);

Page 34: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 33

Author: Juri Calleri

Version: 2

$subject = "$f_notify_type Service: $f_host_name/$f_serv_desc

[$f_serv_state]";

$from ="monitoring@localhost";

$body = "<html><body><table border=0 width='98%' cellpadding=0

cellspacing=0><tr><td valign='top'>\r\n";

$body .= "<table border=0 cellpadding=0 cellspacing=0 width='97%'>";

$body .= "<tr bgcolor=$f_color><td width='140'><b><font

color=#ffffff>Notification:</font></b></td><td><font ";

$body .= "color=#ffffff><b>$f_notify_type

[$f_serv_state]</b></font></td></tr>\r\n";

$body .= "<tr bgcolor=#eeeeee><td><b>Service:</b></td><td><font

color=#0000CC><b>$f_serv_desc</b></font></td></tr>\r\n";

$body .= "<tr bgcolor=#fefefe><td><b>Server:</b></td><td><font

color=#005500><b>$f_host_alias</b></font></td></tr>\r\n";

$body .= "<tr bgcolor=#eeeeee><td><b>Address:</b></td><td><font

color=#005555><b>$f_host_address</b></font></td></tr>\r\n";

$body .= "<tr bgcolor=#fefefe><td><b>Date/Time:</b></td><td><font

color=#005555>$f_long_date</font></td></tr>\r\n";

$body .= "<tr bgcolor=#eeeeee><td><b>More Info:</b></td><td><a

href='https://149.202.109.84/nagios/'>Nagios home

page</a></td></tr>\r\n";

$body .= "<tr bgcolor=#fefefe><td><b>Additional

Info:</b></td><td>$f_serv_output</td></tr>\r\n";

$body .= "<tr bgcolor=#eeeeee><td><b>Extra

Info:</b></td><td>$extra_info</td></tr>\r\n";

$body .= "<tr bgcolor=#fefefe><td><b>State Duration:</b></td><td><font

color=#CC0000><b>$f_duration</b> mins.</font></td></tr> \r\n";

$body .= "<tr bgcolor=#eeeeee><td><b>Service

ExecTime:</b></td><td><font

color=#CC0000><b>$f_exectime</b></font></td></tr></table>\r\n";

$body .= "</td><td valign='top'><table border=0 cellpadding=0

cellspacing=0 width=250><tr bgcolor=#000055><td><b> \r\n";

$body .= "<font color=#FFFFFF>Summary</font></b></td><td>.</td></tr>

\r\n";

$body .= "<tr bgcolor=#f6f6ff><td>Total Service Warnings: </td><td>

$f_totwarnings</td></tr>\r\n";

$body .= "<tr bgcolor=#fffef6><td>Total Service Critical: </td><td>

$f_totcritical</td></tr>\r\n";

$body .= "<tr bgcolor=#f6f6ff><td>Total Service Unknowns: </td><td>

$f_totunknowns</td></tr>\r\n";

$body .= "</table></td></tr></table><br/>\r\n";

$body .= "</body></html> \r\n";

$headers = "From: $from\r\n";

$headers .= "Content-type: text/html\r\n";

/* Send eMail Now... */

$m_true = mail($f_to, $subject, $body, $headers);

echo $m_true;

Page 35: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 34

Author: Juri Calleri

Version: 2

?>

Edit the string $from ="monitoring@localhost"; in this sample too.

Be careful here changing the email account, this is not a real email and the notification sent by

Nagios may arrive to the spam folder of your inbox. Apply a filter to deliver all the emails from

this sender anyway.

Page 36: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 35

Author: Juri Calleri

Version: 2

10 USEFUL PLUGINS

10.1 TOP 4 PLUGINS TO DOWNLOAD

check_log3.pl

Fail2ban is a good tool to constantly monitor the system’s log and apply a ban to those

unauthorized clients that are trying to access the system, it can be ssh or postfix or any other pre-

configured service.

The check_log3.pl plugin works great and the nrpe client contains this string:

command[check_log_fail2ban]=/usr/local/nagios/libexec/check_log3.pl -l

/var/log/fail2ban.log -p 'Ban' -w 3 -c 5

Which reads the last row of the log file and search the word ‘Ban’, if found more than 3 times the

plugin goes to a warning state, when more than 5 it is critical.

fail2ban would have the external IP addresses banned already and Nagios will send an alert email

telling the administrator what happened.

check_service.sh

You may want to check the status of the services too, not only the system health.

This plugin adapts easily to any service on your system, its configuration is simple too and should

the plugin not match the service’s response, the user can add its own “definition”.

It was the case with the postgres database, the plugin did not have the definition for the

status of this service, also, it did lack the performance_data reporting.

This modification at the plugin enabled both:

vi /usr/local/nagios/libexec/check_service.sh

[ reach the switch/case and add your service’s response message ]

# Used for postgres db

*[oO]nline*)

echo "$STATUS_MSG |online=1"

exit $OK

;;

*[dD]own*)

echo "$STATUS_MSG |online=0"

exit $CRITICAL

;;

Everything after the “|” is what is sent as performance_data.

In this example, the command service postgresql status returns:

9.3/main (port 5432): online

The switch/case contain the [oO]nline when the service is online and the [dD]own when it is

not.

The performance_data will graph on a XY axis, the values 1 and 0 according to the service

status.

Page 37: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 36

Author: Juri Calleri

Version: 2

check_linux_stats.pl

This plugin checks the health of your system: cpu, memory, disk, disk i/o, load, network usage,

open files and many other.

Unfortunately, there is a bug which needs to be corrected, it affects the string the client sends to

the server about the CPU status.

To correct it:

vi /usr/local/nagios/libexec/check_linux_stats.pl

[ replace on line 62 ]

if ($o_context){

[ with ]

elsif ($o_context){

Remember though, that you would want to show the graph for the CPU, the next plugin suits best

this task of checking the cpu status.

check_cpu.py

Does what it says, no bugs and comes already with the template for pnp4nagios!

Just remember to rename the php template file after the nrpe command:

check_nrpe_cert_check_cpu.php

Your client nrpe will have this command:

command[check_cpu]=/usr/local/nagios/libexec/check_cpu.py -w 75 -c 99

Warning at 75% and critical at 99%.

Page 38: [IT DOCUMENT] · Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2. Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 1 Author: Juri Calleri Version: 2 TABLE

Title: Installing Nagios 4 with SSL certificate of Nrpe 2.16RC2 37

Author: Juri Calleri

Version: 2

11 USEFUL NAGIOS COMMANDS

11.1 CHECK VALIDITY OF THE SCRIPT

/usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg

11.2 START NAGIOS SERVICE

/usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg

[ or ]

service nagios start

11.3 RELOAD THE SERVICE

/etc/rc.d/init.d/nagios reload

[ or ]

service nagios reload

11.4 STOP THE SERVICE

/etc/rc.d/init.d/nagios stop

[ or ]

service nagios stop

11.5 RESTART NAGIOS AND XINETD

/etc/rc.d/init.d/nagios restart

/etc/init.d/xinetd restart

[ or ]

service nagios restart

service xinetd restart