pandora fms: advanced log parser
DESCRIPTION
This advanced plugin allow you to monitor logs easily, with more options than the default parser included in the agents. For more information visit the following webpage: http://pandorafms.com/index.php?sec=Library&sec2=repository&lng=en&action=view_PUI&id_PUI=297TRANSCRIPT
Pandora FMSAdministrator's Manual
LogParser Monitoring
Administrator's Manual Monitorización LogParser
© Artica Soluciones Tecnológicas 20052012
Indice1Changelog...........................................................................................................................................32Introduction........................................................................................................................................43Requirements......................................................................................................................................54Compatibility Matrix .........................................................................................................................65Software Agent Modules generates....................................................................................................76Instalation...........................................................................................................................................87Monitoring..........................................................................................................................................9
7.1.General Parameters..................................................................................................................107.1.1.include..............................................................................................................................107.1.2.index_dir..........................................................................................................................107.1.3.logfile...............................................................................................................................10
7.2.Log's specific parameters ........................................................................................................107.2.1.log_begin y log_end.........................................................................................................107.2.2.log_module_name............................................................................................................107.2.3.log_description.................................................................................................................107.2.4.log_type............................................................................................................................117.2.5.log_rotate_mode...............................................................................................................117.2.6.log_force_readall.............................................................................................................117.2.7.log_location_exec............................................................................................................117.2.8.log_location_filename......................................................................................................11
7.3.Parametros específicos de la regexp.......................................................................................117.3.1.log_regexp_begin y log_regexp_end...............................................................................117.3.2.log_regexp_rule...............................................................................................................117.3.3.log_regexp_severity.........................................................................................................127.3.4.log_regexp_message........................................................................................................127.3.5.log_regexp_action............................................................................................................12
1 CHANGELOG
Date Author Change Version
02/03/11 Sancho First Version v1r1
22/11/12 Mario P. Second Revision v1r2
Page 3
2 INTRODUCTION
This document describes the generic logs monitoring based in Enterprise parsing logs plugin,
different that OpenSource plugin .This plugin is designed to work with version 3.2.1 or higher.
Page 4
3 REQUIREMENTS
The plugin has the requirements to work correctly:
• Create settings in a configuration file, which the plugin has access. (passed as parameter).
• You can write temporary files (for every log analyzed) to store the last position reading,
inode or md5 signature (for identification of rotated). The default directory is / tmp but this
is a parameter that can be specified in the configuration file.
• Can read the files to process with the user that runs Pandora, or call a script which in turn
call the plugin with all parámeters, so he can read log completly. If you use an external
script, this will have to have permissions to the plugin will generate its index files (see
above)
Page 5
4 COMPATIBILITY MATRIX The agent compatibility matrix is the following:
Systems where it has been tested • Linux (SUSE, Debian, Ubuntu...)
Systems where it should work
• Solaris (con Perl 5.8)• HPUX (con Perl 5.8)• AIX (con Perl 5.8)• Windows
Page 6
5 SOFTWARE AGENT MODULES GENERATES
It will create a module for each parameter that you specify in the configuration file. Config_file is
needed for execution.
The plugin is configured by an external configuration file. This configuration file has a number of
“general” parameters, a series of specific parameters for each log and a set of specific parameters
for each block of regular expression.
Page 7
6 INSTALATION
Copy the plugins to the agent plugin directory, distribute it through file collections or copying it in
the pandora agent folder. Do the same with the additional files that they need. The call from the
agent will be similar to this, but using the paths where the plugin and the list would be installed.
For example:
module_plugin perl /var/opt/PandoraFMS/etc/pandora/plugins/pandora_logparser.pl /var/opt/PandoraFMS/etc/pandora/collections/fc_23/log_example.conf
Page 8
7 MONITORING
The plugin is configured by an external configuration file. This configuration file has a number of
“general” parameters, a series of specific parameters for each log, and a set of specific parameters
for each block of regular expression.
In order to understand each element, following is a sample configuration file:
# Include, to load extenal/aditional configuration files # include /tmp/my_other_configuration.conf
# Directory where temporal indexes will be stored (/tmp by default) #index_dir /tmp
# Log problems with the logparser, (/tmp/pandora_logparser.log by default) #logfile /tmp/pandora_logparser.log
log_begin
log_module_name errores_apache
# This force to process all the log at the beginning log_force_readall
#log_location_exec /tmp/miscript.sh | cut -f 2 log_location_file /var/log/apache2/error_log
log_description This is a nice sample of how powerful is the new logparser
# log rotation detection mode (md5 or inode change), inode by default # log_rotate_mode md5 # log_rotate_mode inode
#log_type return_lines log_type return_ocurrences #log_type return_message
log_regexp_begin log_regexp_rule Critical - ($1)\-($2) log_regexp_rule Critical - ($1)
#log_regexp_severity NORMAL #log_regexp_severity WARNING log_regexp_severity CRITICAL log_return_message Encontrado error CRITICO en bloque $1 seccion $2 log_action <mycommand> log_regexp_end
log_regexp_begin log_regexp_rule Error -($1)\-($2) [0-9a-zA-Z]* log_regexp_severity WARNING log_return_message Otro bonito texto de error log_regexp_end
log_regexp_begin log_regexp_rule File\sdoes\snot\sexist log_regexp_severity WARNING log_regexp_end
log_end
Page 9
log_begin log_force_readall
log_module_name hits_apache log_location_file /var/log/apache2/access_log log_description Access log from Apache, we will get the integria access log_type return_lines
log_regexp_begin log_regexp_rule pandora\.css log_regexp_severity WARNING log_return_message Dispongo de barcos log_regexp_end
log_end
7.1. General Parameters
7.1.1. includeMakes a call to another configuration file. You can nest without limit, and its load order is
sequence. It is important to call files with absolute paths.
7.1.2. index_dirUse this directory to store the index files. The plugin should be able to write and read in the
directory.
7.1.3. logfilePlugin's logfile.
7.2. Log's specific parameters
7.2.1. log_begin y log_endSet marks of the beginning and end of a file definition logparser.log
7.2.2. log_module_nameModule name generated by the plugin.
7.2.3. log_descriptionModule description referring to log file.
Page 10
7.2.4. log_typeLog module type, can be of three types:
• return_ocurrences: Returns a numeric data with the number of occurrences.
• return_lines: Returns the log lines that do match.
• return_message: Returns a message specified by the configuration file.
7.2.5. log_rotate_modeCan be of inode type or md5 type. This is the type detection is done to know if a log is rotated or
not.
7.2.6. log_force_readallWhen this token is present, the log parser processes all the log from the beginning if you have not
already done (Is the first time I opened or detects a rotation). NOTE: You can generate large
volumes of data.
7.2.7. log_location_execExecutes the specified command to obtain the name (absoluto!) file to be processed.
7.2.8. log_location_filenameSpecific the log name (absoluto) file to process.
7.3. Parametros específicos de la regexp
7.3.1. log_regexp_begin y log_regexp_endSet marks of the beginning and end of a regular expression definition for the definition of the log
file in which they are.
7.3.2. log_regexp_ruleDefine the regular expression. NOTE: do not use markers / / Directly the extended regular
expression (Perl type). Examples:
File\sdoes\snot\sexist → Find “File does not exist”
[0-9]*\serrores → Find strings “043 errores”
Page 11
7.3.3. log_regexp_severityIt sent in the XML a sternness, can be WARNING, CRITICAL or NORMAL (in capital letters). Is
optional.
7.3.4. log_regexp_messageText that was sending to find at least one occurrence (if it located several only send a message).
You can use the switches $ 1 .. $ 2 for fields previously identified with a regular expression to do
search field syntax ()→
7.3.5. log_regexp_actionCommand that executes to find at least one occurrence (if it located several run only once).
When defining a log can define several blocks of regular expressions. Each regular expression block may
also have several regular expressions. In the case of multiple matches, it will count each occurrence, but
only send a message or run an action. Should be defined several, will run to make the final "match".
Page 12