monitoring with the sl1 agent (version 10.2.0)

88
Monitoring with the SL1 Agent SL1 version 10.2.0

Upload: others

Post on 20-Apr-2022

6 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Monitoring with the SL1 Agent (Version 10.2.0)

Monitoring with the SL1 AgentSL1 version 10.2.0

Page 2: Monitoring with the SL1 Agent (Version 10.2.0)

Table of Contents

Introduction to the SL1 Agent 4What is the SL1 Agent? 5How the SL1 Agent Works in Different Environments 5What System Vital Metrics Can the SL1 Agent Collect? 7

Metric Descriptions 7Supported Data Collection Methods for Monitoring Windows 9Supported Data Collection Methods for Monitoring Linux 10Extensible Collection 11

Agent PowerPacks 11Agent-Compatible PowerPacks 13

Windows Devices 13Java Management Extensions (JMX) Resources 14

Agent Architecture 14SL1 Distributed Architecture 14SL1 Extended Architecture 16

Installing the SL1 Agent 18Installing an Agent from the Agents Page 19

Installing a Linux Agent 20Using Agents with SELinux 21

Installing a Windows Agent 22Installing an AIX Agent 24Installing a Solaris Agent 25

Configuring an SL1 Agent to Use a Proxy Server Connection 26Configuring the Linux Agent to Use a Proxy 26

Installing and Configuring with the Proxy Settings 26Configuring the Windows Agent to Use a Proxy 27

Installing and Configuring the Agent with the Proxy Settings 28Upgrading the Agent 29Stopping the Agent 30Uninstalling the Agent 30

Uninstalling a Linux Agent 31Uninstalling a Windows Agent 31

Installing an Agent from the Device Manager Page 31Installing the Linux Agent from the Device Manager Page 33

Updating an Agent on a Linux System 34Uninstalling an Agent on a Linux System 34

Installing the Windows Agent from the Device Manager Page 35Uninstalling the Agent on a Windows System 35

Configuring the SL1 Agent to Work with Phone Home Collectors 35Remotely Installing the SL1 Agent 38

Remotely Installing the Linux Agent 38Remotely Installing the SL1 Windows Agent 43

Configuring the SL1 Agent 47Configuring Agent Monitoring Based on SL1 Architecture 48Configuring Agent Settings from the Device Investigator Page 49Configuring Extensible Collection 52Configuring an SL1 Agent on the Device Manager Page 54

Adding the "SL Agent" Column to the Device Manager Page 55Configuring Agent Settings on a Device 56

Monitoring Logs Using the SL1 Agent 58

Page 3: Monitoring with the SL1 Agent (Version 10.2.0)

What is a Log File Monitoring Policy? 59Viewing the List of Log File Monitoring Policies 59

Filtering the List of Log File Monitoring Policies 60Creating a Log File Monitoring Policy 61Aligning a Log File Monitoring Policy to Devices 63

Unaligning Log File Monitoring Policies from Devices 66Editing a Log File Monitoring Policy 66Deleting Log File Monitoring Policies 67Viewing the List of Log File Monitoring Policies and Aligned Devices 67

Filtering the List of Log File Monitoring Policies and Aligned Devices 68Creating an Event Policy for Agent Logs 68Creating an Event Policy for Agent Logs in the Classic User Interface 73

Agent Troubleshooting 76Was the Agent Download Successful? 78Is the Agent Process Running? 78Is the Agent Configuration Valid? 79Has SL1 Discovery Completed? 79Is the Agent Able to Upload Data? 80

Check the Agent Upload Directory 80Run the Agent in Debug Mode (Linux) 81

Is SL1 Receiving Agent Data? 81Can SL1 Process Agent Data? 82Is the Number of Processes Inconsistent with Other Applications? 82Does MySQL on the Collector Unit Have a Connection Issue? 83Troubleshooting Examples 83

Example /var/log/streamer_prime/streamer_prime.log for successful discovery 83Example /var/log/uwsgi/streamer.log for successful discovery in the distributed architecture 84Save incoming data for a specific device ID in the distributed architecture 84Save incoming data for a specific device ID in the SL1 Extended Architecture 84

Additional Troubleshooting Situations and Best Practices 84Agent Communication with SL1 85

Return Codes 85Sub-codes 86

Page 4: Monitoring with the SL1 Agent (Version 10.2.0)

Chapter

1Introduction to the SL1 Agent

Overview

This chapter describes the SL1 agent and provides instructions for viewing device and interface data collected bythe agent.

Use the following menu options to navigate the SL1 user interface:

l To view a pop-out list of menu options, click the menu icon ( ).

l To view a page containing all the menu options, click the Advanced menu icon ( ).

This chapter includes the following topics:

What is the SL1 Agent? 5

How the SL1 Agent Works in Different Environments 5

What System Vital Metrics Can the SL1 Agent Collect? 7

Agent PowerPacks 11

Agent Architecture 14

4

Page 5: Monitoring with the SL1 Agent (Version 10.2.0)

5

What is the SL1 Agent?

The SL1 agent is a program that you can install on a device monitored by SL1. The SL1 agent collects data fromthe device and pushes that data back to SL1.

Similar to a Data Collector or Message Collector, the SL1 Agent collects data about infrastructure andapplications.

The agent can be configured to communicate with either the Message Collector or Compute Cluster.

How the SL1 Agent Works in Different Environments

There are two supported "generations" of the SL1 agent that you can use to gather data: Gen 1 and Gen 3. Theagent itself is the same from one generation to the next; the only difference between the generations is theenvironment where the agent is used.

In brief, aGen 1 agent uses a distributed SL1 environment to upload data directly to a Message Collector (MC),while aGen 3 agent uses the SL1 Extended Architecture to upload data to the "Streamer" service running on theSL1 Compute Node cluster.

The following list provides more details about how SL1 uses the different generations of the SL1 agent:

l Gen 1 agent: Introduced in SL1 version 8.2.0, the Gen 1 agent uses a distributed environment to uploaddata directly to a Message Collector (MC). Next, Dynamic Applications running on a Data Collector querydata from the Message Collector with an API. The Data Collector (DC) then sends the data to the SL1Database Server:

The Gen 1 agent provides limited infrastructure health reporting, including Log File Monitoring, Processes,and System Vitals like Availability, CPU Usage, Memory Usage, and Disk Usage. This agent is sometimescalled the "P0" agent.

What is the SL1 Agent?

Page 6: Monitoring with the SL1 Agent (Version 10.2.0)

How the SL1 Agent Works in Different Environments

l Gen 2 agent: This version of the agent has been deprecated.

l Gen 3 agent: Introduced in SL1 version 8.12.0, the Gen 3 agent uses the SL1 Extended Architecture toupload data through a load balancer to the "Streamer" service running on the SL1 Compute Node cluster.The Streamer service then forks and forwards data to other services, and eventually some services will storeparsed data in the SL1Database Server:

The Gen 3 agent provides full infrastructure health reporting, including system vitals (file system, networkinterface, and Windows service data), log monitoring, and optionally allows you to align PowerShellDynamic Applications to your device. Also, the Dynamic Applications in the ScienceLogic: AgentPowerPack are automatically aligned with the device with the Gen 3 agent installed on it.

If you are already running the Gen 1 agent in a distributed environment and you upgrade your SL1 system to usethe SL1 Extended Architecture, any existing devices monitored by SL1 agents will work the same as before(streaming data to a Message Collector). However, if you upgrade to the SL1 Extended Architecture,ScienceLogic recommends that you start streaming your agents to the Compute Node cluster instead of to aMessage Collector.

To identify the version of the agent installed on a device:

l Gen 1. On the Devices > Device Investigator > [Settings] tab for that device, the Collection Pollerfield displays the name of the collector group that includes the Message Collector used by the agent. On theDevice Manager page (Devices > Device Manager), the Collection Group column displays the name ofthe collector group that includes the Message Collector used by the agent.

l Gen 3. On the Devices > Device Investigator > [Settings] tab for that device, the Collection Pollerfield displays the label Agents. On the Device Manager page (Devices > Device Manager), theCollection Group column displays the label Agents.

6

Page 7: Monitoring with the SL1 Agent (Version 10.2.0)

7

The following table provides an overview of the features available when using the Gen 1 or the Gen 3 agent:

Product Capability Gen 1 Agent Gen 3 Agent

System Vitals

Availability and Uptime Yes Yes

CPU, Memory, File Systems, and Network Interface CPU and Memory only Yes

Processes and Windows Services Processes only Yes

Installed Software No Yes

Log Monitoring

Event Logs Yes Yes

Syslog Yes Yes

Text Logs Yes Yes

Extensible Collection

PowerShell No Yes

JMX No Yes

For a detailed list of the system vital metrics you can monitor with the SL1 agent, seeWhat System Vital MetricsCan the SL1 Agent Collect?

For a current list of supported operating systems and host system requirements for the SL1 agent, see the SystemRequirements for the Agent page at the ScienceLogic Support Site.

NOTE: Users who are running version 102 or later of theMicrosoft: Windows Server PowerPack can collectdata via the SL1 agent. For more information, see theMonitoring Windows Systems withPowerShellmanual.

What System Vital Metrics Can the SL1 Agent Collect?

The following sections describe the system vitals that can be collected with SL1 and with the SL1 Agent, includingdefinitions of each metric type and the collection methods that are and are not supported for each.

Metric Descriptions

The following table describes the system vital metrics that can be collected with SL1 and the SL1 Agent:

What System Vital Metrics Can the SL1 Agent Collect?

Page 8: Monitoring with the SL1 Agent (Version 10.2.0)

What System Vital Metrics Can the SL1 Agent Collect?

Metric Type Description

Availability Performance The ability to communicate with the managed entity or device.

File Systems Configuration The configuration of the file systems found within a managed entity that caninclude attributes like name, size, and type.

File Systems Performance Time series data associated with file system utilization that can includemetrics like free space, size, and usage percentage.

InstalledSoftware

Configuration The software found on a managed entity that can include attributes likename, version, and installation date.

NetworkInterfaces

Configuration The configuration of the network interface found within a managed entitythat includes attributes like MAC address, IP address, position, and speed.

NetworkInterfaces

Performance Time series data associated with physical memory utilization that includesmetrics like inbound and outbound utilization, number of errors, anddiscard and usage percentage.

Physical Memory Configuration The configuration of the physical memory found within a managed entitythat can include attributes like memory size.

Physical Memory Performance Time series data associated with physical memory utilization that caninclude metrics like memory used.

Ports Configuration The ports discovered on a managed entity.

Ports Performance Time series data associated with port availability.

Ports (Illicit) Performance An analysis of ports. When a port from the illicit port list is found on amanaged system, the system will trigger an event indicating an illicit porthas been found.

Processes Configuration The processes found on a managed entity that can include attributes likename, process ID (PID), and state.

Processes Performance Time series data associated with process performance that can includemetrics like availability percentage.

Processor Configuration The configuration of the processor found within a managed entity that caninclude attributes like number of cores, processor model, processor speed,cache size, and CPU ID.

Processor Performance Time series data associated with processor utilization that can includemetrics like CPU idle time, CPU wait time, and overall CPU time.

Restarts Performance An analysis of uptime. When uptime is less than 15 minutes, the systemtriggers an event indicating the system was restarted.

SSL Certificates Configuration The certificates found on a managed system.

SSL Certificates Performance An analysis of certificate expiration date. The system will trigger an eventwhen certificates are nearing expiration.

8

Page 9: Monitoring with the SL1 Agent (Version 10.2.0)

9

Metric Type Description

Uptime Performance The timespan since the managed entity was last initialized.

Virtual Memory(Swap)

Configuration The configuration of the virtual memory found within a managed entity.

Virtual Memory(Swap)

Performance Time series data associated with virtual memory utilization.

WindowsServices

Configuration The services found on a managed entity that can include attributes likename and state.

WindowsServices

Performance Time series data associated with service performance that can includemetrics like availability percentage.

Supported Data Collection Methods for Monitoring Windows

The following table describes which methods of data collection are supported when running SL1 and the SL1Agent on monitored Windows systems:

Metric Type

Agentless Agent-Based

SNMP WMI PowerShell Gen-01 Gen-03

Availability Performance Yes Yes Yes Yes Yes

File Systems Configuration Yes Some Yes Some Yes

File Systems Performance Yes Some Yes Some Yes

Installed Software Configuration Yes No Yes No Yes

Network Interfaces Configuration Yes Some Yes Some Yes

Network Interfaces Performance Yes Some Yes Some Yes

Physical Memory Configuration Yes Yes Yes Yes Yes

Physical Memory Performance Yes Yes Yes Yes Yes

Ports Configuration Yes No Yes Yes No

Ports Performance Yes No Yes Yes No

Ports (Illicit) Performance Yes No Yes Yes No

Processes Configuration Yes Some Yes Yes Yes

Processes Performance Yes No Yes Yes Yes

What System Vital Metrics Can the SL1 Agent Collect?

Page 10: Monitoring with the SL1 Agent (Version 10.2.0)

What System Vital Metrics Can the SL1 Agent Collect?

Metric Type

Agentless Agent-Based

SNMP WMI PowerShell Gen-01 Gen-03

Processor Configuration Yes Yes Yes Yes Yes

Processor Performance Yes Yes Yes Yes Yes

Restarts Performance Yes No Yes Yes Yes

SSL Certificates Configuration Yes No No No No

SSL Certificates Performance Yes No No No No

Uptime Performance Yes No Yes Yes Yes

Virtual Memory (Swap) Configuration Yes Yes Yes Yes Yes

Virtual Memory (Swap) Performance Yes Yes Yes Yes Yes

Windows Services Configuration Yes Some Yes No Yes

Windows Services Performance Yes Some Yes No Yes

Supported Data Collection Methods for Monitoring Linux

The following table describes which methods of data collection are supported when running SL1 and the SL1Agent on monitored Linux systems:

Metric Type

Agentless Agent-Based

SNMP SSH Gen-01 Gen-03

Availability Performance Yes Yes Yes Yes

File Systems Configuration Yes Yes Some Yes

File Systems Performance Yes Yes Some Yes

Installed Software Configuration Yes No No Yes

Network Interfaces Configuration Yes Yes Some Yes

Network Interfaces Performance Yes Yes Some Yes

Physical Memory Configuration Yes Yes Yes Yes

Physical Memory Performance Yes Yes Yes Yes

Ports Configuration Yes Yes Yes No

10

Page 11: Monitoring with the SL1 Agent (Version 10.2.0)

11

Metric Type

Agentless Agent-Based

SNMP SSH Gen-01 Gen-03

Ports Performance Yes Yes Yes No

Ports (Illicit) Performance Yes Yes Yes No

Processes Configuration Yes Yes Yes Yes

Processes Performance Yes Yes Yes Yes

Processor Configuration Yes Yes Yes Yes

Processor Performance Yes Yes Yes Yes

Restarts Performance Yes Yes Yes Yes

SSL Certificates Configuration Yes No No No

SSL Certificates Performance Yes No No No

Uptime Performance Yes Yes Yes Yes

Virtual Memory (Swap) Configuration Yes Yes Yes Yes

Virtual Memory (Swap) Performance Yes Yes Yes Yes

Windows Services Configuration N/A N/A N/A N/A

Windows Services Performance N/A N/A N/A N/A

Extensible Collection

In addition to the capabilities listed above, you can use the SL1 agent for "extensible collection", where you alignthe agent with Dynamic Applications to gather metrics and attributes from other infrastructures and applications.

The SL1 Extended Architecture supports aligning PowerShell Dynamic Applications to devices monitored by theSL1 Windows Agent. The SL1 Extended Architecture supports aligning JMX Dynamic Applications to devicesmonitored by the SL1 Linux Agent.

For more information, see Configuring Extensible Collection.

Agent PowerPacks

SL1 includes two PowerPacks that can be used to collect agent-based system configuration and performancedata: the ScienceLogic: Agent PowerPack and the Host Agent PowerPack.

Agent PowerPacks

Page 12: Monitoring with the SL1 Agent (Version 10.2.0)

Agent PowerPacks

Both PowerPacks are installed by default on your SL1 system, and they include the following features:

l Dynamic Applications that collect configuration data and performance metrics from devices that are usingagent-based collection

l Event Policies and alerts that are triggered when devices that are using agent-based collection meet certainstatus criteria

The ScienceLogic: Agent PowerPack collects agent-based data for devices on SL1 systems running on the SL1Extended Architecture (Gen 3 agents). This PowerPack contains two Dynamic Applications:

l The "ScienceLogic Agent: System Configuration" Dynamic Applications collects the following data:

o CPU

o CPU Information

o CPUs

o Hardware Totals

o Memory

o Speed (MHz)

o Swap Capacity

l The "ScienceLogic Agent: System Performance" Dynamic Applications collects the following data:

o CPU Name

o CPU Utilization

o CPU Utilization Breakdown

o Disk Average Queue Length

o Disk IO Utilization

o Disk Name

o Memory Utilization

o Network Read

o Network Write

o Sample Time

o Swap Utilization

The Host Agent PowerPack, which collects agent-based data for devices on SL1 systems running on adistributed architecture (Gen 1 agents). This PowerPack contains two Dynamic Applications:

l The "Host Agent: System Config" Dynamic Applications collects the following data:

o CPU

o CPU Information

o CPUs

12

Page 13: Monitoring with the SL1 Agent (Version 10.2.0)

13

o Disk

o Disk Information

o Disk Space

o Disks

o Hardware Totals

o Memory

o Size

o Speed (MHz)

l The "Host Agent: System Perf" Dynamic Applications collects the following data:

o CPU Name

o CPU Utilization

o CPU Utilization Breakdown

o Disk Average Queue Length

o Disk Name

o Disk Utilization

o Memory Utilization

o Network Read

o Network Write

o Sample Time

o Storage Available

o Storage Name

o Storage Total

o Storage Utilization

NOTE: Because the ScienceLogic: Agent PowerPack is required to collect data from devices that are usingagent-based collection, SL1 does not enable you to delete or modify this PowerPack.

Agent-Compatible PowerPacks

In addition to the ScienceLogic: Agent PowerPack and the Host Agent PowerPack, there are several other Agent-compatible PowerPacks you can use to collect data from specific device types.

Windows Devices

The following PowerPacks include the SL1 Agent PowerShell Default credential and SL1 Agent device template,which you can use to execute the SL1 Agent on Windows devices with PowerShell:

Agent PowerPacks

Page 14: Monitoring with the SL1 Agent (Version 10.2.0)

Agent Architecture

l Microsoft: Windows Server

l SL1 Agent Templates for Microsoft PowerPacks, which includes templates for the following:

o Microsoft: DHCP Server

o Microsoft: DNS Server

o Microsoft: Exchange Server

o Microsoft: IIS Server

o Microsoft: Lync Server

o Microsoft: SharePoint Server

o Microsoft: SQL Server

o Microsoft: Windows Server

NOTE: For more information, see theMonitoring Windows with PowerShellmanual.

Java Management Extensions (JMX) Resources

You can also use the JMX Base Pack *BETA* PowerPack to monitor JMX resources with the SL1 agent.

For more information, see theMonitoring Java Management Extensions (JMX)manual.

Agent Architecture

The following sections describe how the SL1 agent works in a distributed architecture and in the SL1 ExtendedArchitecture.

SL1 Distributed Architecture

In a distributed architecture, the SL1 Agent collects data from the device on which it is installed and transfers thatdata to a Message Collector in an SL1 system using the HTTPS protocol. The Data Collector on which theDynamic Applications and collection processes run then poll the Message Collector using the HTTPS protocol totransfer data to SL1.

TCP port 443 must be open between the Message Collector and the device on which an agent is installed.

In a distributed architecture, the SL1 agent requires a standalone, dedicated Message Collector. The MessageCollector does not need to be dedicated to agent usage, but the Message Collector cannot be a Data Collectorthat also performs message collection

14

Page 15: Monitoring with the SL1 Agent (Version 10.2.0)

15

NOTE:Message Collectors that process data from the SL1 agent have different system requirements thanMessage Collectors that do not process data from the SL1 agent. For more information about thesystem requirements when running SL1 agents in a distributed architecture, see the SystemRequirements page at the ScienceLogic Support Site.

The diagram below shows the collection layer of a distributed system containing both Data Collectors andMessage Collectors in which the SL1 Agent is installed on a managed device.

Agent Architecture

Page 16: Monitoring with the SL1 Agent (Version 10.2.0)

Agent Architecture

SL1 Extended Architecture

In the SL1 Extended Architecture, an SL1 agent collects data from the device on which it is installed and sendsthat data to a Load Balancer in front of a Compute Cluster. The Compute Cluster transforms the data and storeshigh-volume performance data in the Storage Cluster and other performance and configuration data in theDatabase Server.

If required, agents can use an HTTP proxy server as an intermediate step in sending data to SL1.

In the diagram below:

l The SL1 agent collects data from managed devices and sends the data to the Load Balancer and ComputeNode cluster for processing.

l The optionalMessage Collector collects asynchronous traps and syslog messages and sends them to theDatabase Server.

l The Data Collector collects data from managed devices and sends the data to the Load Balancer andCompute Node cluster for processing and then storage.

Using an SL1 agent in the SL1 Extended Architecture provides more configuration and performance data thanusing an SL1 agent in a distributed architecture. This additional data includes system vitals, log data, andextensible collection.

16

Page 17: Monitoring with the SL1 Agent (Version 10.2.0)

17

NOTE: For more information about the system requirements when running SL1 agents in an extendedarchitecture, see the System Requirements page at the ScienceLogic Support Site.

Agent Architecture

Page 18: Monitoring with the SL1 Agent (Version 10.2.0)

Chapter

2Installing the SL1 Agent

Overview

This chapter describes how to install, upgrade, and uninstall SL1 agents for Windows, Linux, AIX, and Solarisoperating systems. All agent installations, upgrades, and uninstallations require command-line interface accessto the targeted device.

l If you use the SL1 Extended Architecture (which includes Compute Nodes, Storage Nodes, and aManagement node), install the agent from the Agents page (Devices > Agent). For details, see Installingan Agent from the Agents Page.

l If you use a distributed SL1 architecture (where the SL1 Agent collects and transfers data to a MessageCollector), install the agent from the Device Manager page (Devices > Device Manager). For details, seeInstalling an Agent from the Device Manager Page.

NOTE: If you are using PhoneHome Message Collectors in a distributed architecture, see Configuring theSL1 Agent to Work with PhoneHome Collectors before you install the SL1 agent.

Use the following menu options to navigate the SL1 user interface:

l To view a pop-out list of menu options, click the menu icon ( ).

l To view a page containing all the menu options, click the Advanced menu icon ( ).

This chapter includes the following topics:

Installing an Agent from the Agents Page 19

Configuring an SL1 Agent to Use a Proxy Server Connection 26

Upgrading the Agent 29

18

Page 19: Monitoring with the SL1 Agent (Version 10.2.0)

19

Stopping the Agent 30

Uninstalling the Agent 30

Installing an Agent from the Device Manager Page 31

Remotely Installing the SL1 Agent 38

Installing an Agent from the Agents Page

When running SL1 on the SL1 Extended Architecture, you can use the Agents page (Devices > Agents) to helpyou install the SL1 agent on a Linux, Windows, AIX, or Solaris device:

TIP: The Agents page appears as an option on the Devicesmenu only when you are using the SL1 ExtendedArchitecture. If your version of SL1 does not have an Agents page, see Installing an Agent from theDevice Manager Page.

NOTE: If you are using an SL1 system with the SL1 Extended Architecture, youmust use the Agents page(Devices > Agents) to install agents, including AIX and Solaris agents. AIX and Solaris agents are notavailable for non-SL1 Extended systems.

Installing an Agent from the Agents Page

Page 20: Monitoring with the SL1 Agent (Version 10.2.0)

Installing an Agent from the Agents Page

You can also upgrade and delete agents from SL1 on the Agents page.

TIP: If you are looking for a very specific set of agents, click the gear icon ( ) to the right of the Search fieldand select Advanced. In this mode you can create an advanced search using "AND" or "OR" for multiplesearch criteria. For more information, see the "Performing an Advanced Search" topic in theIntroduction to SL1manual.

You do not install the agent from the Agents page itself. Instead, the Agents page enables you to gather theinformation or files you need to then install the agent on a particular device.

The instructions on the relevant tab of the Agents page also include command line parameters such asURLFRONT, which is used to define the Streamer service you are using as part of the SL1 Extended Architecture.

If the agent installation is successful, one of the following automatically happens:

l If the primary IP address of the device is not currently monitored by SL1, then SL1 creates a device record forthe device and populates the device record with data provided by the agent. The device record is assigned adevice class based on data reported by the agent.

l If the primary IP address of the device is currently monitored by SL1, the device record for the existing deviceis updated with data provided by the agent.

During initial discovery, the agent returns operating system type and version information to SL1. Based on thisinformation, SL1 assigns the corresponding device classes to a device monitored only by an agent. If a device ismonitored by an agent and via SNMP, the device class assigned by SNMP discovery will take precedence.

Installing a Linux Agent

For a Linux system, the Agents page provides version-specific commands that you will need to run on the Linuxsystem you want to monitor.

NOTE: SL1 supports the use of proxy server connections when using the SL1 agent on Linux systems. Formore information about installing and configuring a Linux agent with a proxy server, seeConfiguring the Linux Agent to Use a Proxy.

NOTE: If you are installing on SELinux, see Using Agents with SELinux.

20

Page 21: Monitoring with the SL1 Agent (Version 10.2.0)

21

To install a Linux agent:

1. On the Agents page (Devices > Agents), click [New Agent]. The Agent Installation page appears:

2. Click the [Linux] tab at top left.

3. From the Select an Organization drop-down list, select an organization for the device on which you areinstalling the agent. The TENANT value in the commands on this page is updated with the SL1Organization ID for the organization you selected.

TIP: The URLFRONT value is the URL for the Streamer service you are using as part of the SL1Extended Architecture.

4. Based on the version of Linux running on the device you want to monitor, run the relevant commands fromthe [Linux] tab on the Linux device (the "target server"). After the commands complete, the Linux agent startsrunning in the background. The device on which you installed the agent appears on the Agents page(Devices > Agents), and the device is also added to the list of devices on the Devices page.

Using Agents with SELinux

The Linux agent supports running in SELinux for CentOS or Red Hat in both permissive and enforcing modes. TheRPM installer for Linux will detect if SELinux is active (permissive/enforcing) upon installation. The agentautomatically creates and installs an appropriate SELinux policy for use with the agent daemon.

With the Linux Agent, a shared library gets injected into other processes. For non-root processes, this sharedlibrary should not generate any errors or audit logs. However, for root processes, this shared library mightgenerate some additional audit logs where some of the shared library functionality is restricted by SELinux. If thisis the case, you can update the agent configuration to exclude pre-loading into these root processes.

Installing an Agent from the Agents Page

Page 22: Monitoring with the SL1 Agent (Version 10.2.0)

Installing an Agent from the Agents Page

Installing a Windows Agent

For a Windows system, the Agents page provides an executable file that you will need to run on the Windowssystem you want to monitor.

NOTE: SL1 supports the use of proxy server connections when using the SL1 agent on Windows systems. Formore information about installing and configuring a Windows agent with a proxy server, seeConfiguring the Windows Agent to Use a Proxy.

To download and install a Windows agent in SL1:

1. On the Agents page (Devices > Agents), click [New Agent]. The Agent Installation page appears.

2. Click theWindows tab at top left:

3. From the Select an Organization drop-down list, select an organization for the new agent. The tenantvalue in the command on this page is updated with the SL1 Organization ID for the organization youselected.

4. Click [Download Agent]. AnOpening SiloAgent-install.exe dialog appears.

5. Save the SiloAgent-install.exe file for installing the agent.

6. Copy the SiloAgent-install.exe file you downloaded to the Windows system on which you want to run theagent. To do so, you can either go to the console of the Windows system or use a utility like WinSCP.

22

Page 23: Monitoring with the SL1 Agent (Version 10.2.0)

23

7. To install the agent, use Windows PowerShell as an Administrator to run the command on the Windowssystem (the command is listed in step 2 on theWindows tab on the Agents page):

.\SiloAgent-install.exe tenant=<organization_id> urlfront=<streamer_URL>

where <organization_id> is Organization ID for the organization you selected in step 3, and<streamer_URL> is the URL for the Streamer service you are using as part of the SL1 ExtendedArchitecture.

For example:

.\SiloAgent-install.exe tenant=1 urlfront=streamer-2020.int.sciencelogic.com

8. When the agent installation completes, the PowerShell window displays "info: install complete", and theWindows agent starts running in the background. The device on which you installed the agent appears onthe Agents page (Devices > Agents), and the device is also added to the list of devices on the Devicespage.

9. To verify that the installation was successful, open the Windows Task Manager or enter the TASKLISTcommand to view running processes. The SiloAgent process will be running on the Windows machine.

Installing an Agent from the Agents Page

Page 24: Monitoring with the SL1 Agent (Version 10.2.0)

Installing an Agent from the Agents Page

Installing an AIX Agent

For an AIX system, the Agents page provides a set of commands that you will need to run on the AIX system youwant to monitor.

NOTE: The AIX agent is a Beta feature in SL1 version 10.1.0.

To download and install an AIX agent:

1. On the Agents page (Devices > Agents), click [New Agent]. The Agent Installation page appears:

2. Click the [AIX] tab.

3. From the Select an Organization drop-down list, select an organization for the device on which you areinstalling the agent. The Tenant value in the commands on this page is updated with the SL1 OrganizationID for the organization you selected.

TIP: The URLFront value is the URL for the Streamer service you are using as part of the SL1 ExtendedArchitecture.

4. Run the relevant commands from the [AIX] tab on the AIX device (the "target server"). After the commandscomplete, the AIX agent starts running in the background. The device on which you installed the agentappears on the Agents page (Devices > Agents), and the device is also added to the list of devices on theDevices page.

24

Page 25: Monitoring with the SL1 Agent (Version 10.2.0)

25

Installing a Solaris Agent

For a Solaris system, the Agents page provides version-specific commands that will need to run on the Solarissystem you want to monitor.

NOTE: The Solaris agent is a Beta feature in SL1 version 10.1.0.

To download and install a Solaris agent:

1. On the Agents page (Devices > Agents), click the [New Agent] button. The Agent Installation pageappears.

2. Click the [Solaris] tab:

3. From the Select an Organization drop-down list, select an organization for the device on which you areinstalling the agent. The Tenant value in the commands on this page is updated with the SL1 OrganizationID for the organization you selected.

TIP: The URLFront value is the URL for the Streamer service you are using as part of the SL1 ExtendedArchitecture.

Installing an Agent from the Agents Page

Page 26: Monitoring with the SL1 Agent (Version 10.2.0)

Configuring an SL1 Agent to Use a Proxy Server Connection

4. Based on the version of Solaris running on the device you want to monitor, run the relevant commands fromthe [Solaris ] tab on the Solaris device (the "target server"). After the commands complete, the Solaris agentstarts running in the background. The device on which you installed the agent appears on the Agents page(Devices > Agents), and the device is also added to the list of devices on the Devices page.

Configuring an SL1 Agent to Use a Proxy Server Connection

The following section covers how you can configure a proxy server connection when using the SL1 agent on Linuxor Windows systems.

Configuring the Linux Agent to Use a Proxy

To configure the Linux agent to use a proxy server connection:

1. Create a configuration file named scilog_proxy.conf and add the following proxy information to the newfile:

Proxy-ServerProxy-PortProxy-UsernameProxy-Password

For example:

Proxy-Server 10.1.1.1Proxy-Port 8080Proxy-Username admin1Proxy-Password passw0rd

NOTE: The agent requires at least the proxy server and proxy port information. If no authentication isrequired, leave the username and password commented out with a "#". If values are presentfor the username and password, the agent will try to use them. The agent currently supportsBasic Authentication.

2. Add the new file to the /etc/scilog directory on the SL1 server. This directory is also where the scilog.conffile resides.

NOTE: If the scilog_proxy.conf file exists, the agent will try to connect to a proxy. If you do not needa proxy, remove or rename this file to prevent the agent from accessing it. Alternatively, youcan leave the parameters blank in the scilog_proxy.conf file.

Instal l ing and Configuring with the Proxy Sett ings

You can set the proxy parameters at install time by including them as parameters for the install command, just asyou do with the Tenant and URLFront values. Use the package name for the package being installed.

For example:

26

Page 27: Monitoring with the SL1 Agent (Version 10.2.0)

27

deb

sudo TENANT=0 URLFRONT=10.1.1.2 proxyserver=10.2.1.1 proxyport=3128 proxyusername=1proxypassword=1 dpkg -i sl_agent_v173.15.deb

rpm

sudo TENANT=0 URLFRONT=10.1.1.2 proxyserver=10.2.1.1 proxyport=3128 proxyusername=1proxypassword=1 rpm -ivh sl_agent_v173.16.rpm

aix

sudo TENANT=0 URLFRONT=10.1.1.2 proxyserver=10.2.1.1 proxyport=3128 proxyusername=1proxypassword=1 rpm -Uvh scilogd-0.173.17-0.aix7.2.noarch.rpm

NOTE: If no proxy is used, you can remove the proxy parameters in the install command. Similarly, if a proxyis used without credentials, remove the proxyusername and proxypassword parameters in theinstall command, and only include the proxyserver and proxyport parameters .

NOTE: Also note that the Tenant and URLFront values can now be passed on the install command for AIX.

Configuring the Windows Agent to Use a Proxy

To configure the Windows agent to use a proxy server connection:

1. Create a configuration file named scilog_proxy.conf and add the following proxy information to the newfile:

Proxy-ServerProxy-PortProxy-UsernameProxy-Password

For example:

Proxy-Server 10.1.1.1Proxy-Port 8080Proxy-Username admin1Proxy-Password passw0rd

NOTE: The agent requires at least the proxy server and proxy port information. If no authentication isrequired, leave the username and password values blank. If values are present for theusername and password, the agent will try to use them. The agent currently supports BasicAuthentication. The agent will discover the type of supported authentication, so there is noneed to specify it in the configuration file.

Configuring an SL1 Agent to Use a Proxy Server Connection

Page 28: Monitoring with the SL1 Agent (Version 10.2.0)

Configuring an SL1 Agent to Use a Proxy Server Connection

2. Add the new file to the C:\Program Files\ScienceLogic\SiloAgent\conf directory on the SL1 server. Thisdirectory is also where the scilog.conf file resides.

NOTE: If the scilog_proxy.conf file exists, the agent will try to connect to a proxy. If you do not needa proxy, remove or rename this file to prevent the agent from accessing it. Alternatively, youcan leave the parameters blank in the scilog_proxy.conf file. If you use the .msi installer, youcan input the parameters in the dialog box for the installer.

When configuring a proxy password with the above procedure, SL1 encodes the proxy password in the scilog_proxy.conf file. The password displays as a seemingly random string of characters.

NOTE: You can manually edit the scilog_proxy.conf file, you can manually enter the proxy password. Theagent will still work with a proxy password in plain text. The agent will detect if the setup scriptencoded the password; otherwise, it assumes the password is in plain text.

Instal l ing and Configuring the Agent with the Proxy Sett ings

You can set the proxy parameters at install time by including them as parameters for the install command, just asyou do with the Tenant and URLFront values.

For example:

.\SiloAgent-install.exe Tenant=0 URLFront=10.1.1.2 proxyserver=10.2.1.1proxyport=3128 proxyusername=1 proxypassword=1

NOTE: If no proxy is used, you can remove the proxy parameters in the install command. Similarly, if a proxyis used without credentials, remove the proxyusername and proxypassword parameters in theinstall command, and only include the proxyserver and proxyport parameters.

NOTE: If no parameters are passed to SiloAgent-install.exe, including Tenant and URLFront, a pop-updialog will appear where you can enter these values. If you do not require proxy credentials, you canleave those values blank.

28

Page 29: Monitoring with the SL1 Agent (Version 10.2.0)

29

Upgrading the Agent

When you have the latest version of the SL1 agent, a check mark icon ( ) appears in theNewest Versioncolumn for that agent. If you do no have the latest version of the agent, an [Upgrade] button appears in theNewest Version column on the Agents page.

To upgrade to the latest version of an agent:

1. On the Agents page (Devices > Agents), locate the agent you want to upgrade:

2. Click the [Upgrade] button for that agent, or click the [Actions] button ( ) and select Upgrade. The agentstarts the upgrade process, which might take up to a minute to complete.

3. Click the [Refresh] button in your browser to ensure that a check mark icon ( ) appears in theNewestVersion column for that agent.

Upgrading the Agent

Page 30: Monitoring with the SL1 Agent (Version 10.2.0)

Stopping the Agent

Stopping the Agent

You can use the "delete" option on the Agents page to stop an agent from running. When you delete an agent,SL1 stops the agent from running on the device, deletes the data gathered by that agent, and removes that agentfrom the Agents page.

NOTE: Using the "delete" option for an agent does not actually remove the agent from the device. As a bestpractice, use the delete process to delete the data gathered by the agent (the uninstallation processdoes not delete this data), and then uninstall that agent, if needed. For uninstallation details, seeUninstalling an Agent. Also, ScienceLogic recommends that you delete the device that had theagent installed on it, or at least remove the agent-related Dynamic Applications, to prevent SL1 fromattempting to use the data the agent gathered from that device.

To delete an agent and its data:

1. On the Agents page (Devices > Agents), locate the agent you want to delete.

2. Click the Actions button ( ) for that agent and select Delete. SL1 stops the agent from collecting data.

NOTE: If you used the Agents page to install an agent, then you should also use the Agents page to deletethat agent. If you attempt to delete the agent using the classic SL1 user interface rather than theAgents page, SL1 deletes only some of the data gathered by the agent, rather than all of the data.

Uninstalling the Agent

When you uninstall an agent, you remove that agent completely from SL1, but you do not lose the data collectedby that agent.

NOTE: After you uninstall the agent, the scilog.conf file remains on the device. If you think you might wantto monitor with that agent again in the future, you should delete the scilog.conf file after you run theuninstallation. 

30

Page 31: Monitoring with the SL1 Agent (Version 10.2.0)

31

Uninstalling a Linux Agent

To uninstall an agent on a Linux system:

1. Log in to the Linux system via the console or SSH as a user that has sudo administrator permissions.

2. For Red Hat, CentOS, and Oracle, execute the following command:

rpm –e scilogd

3. For Debian and Ubuntu, execute the following command:

dpkg -r scilogd

4. Optionally, you can remove the agent configuration directory from the Linux system. The configurationdirectory can be found at:

/etc/scilog (rm -rf /etc/scilog)

Uninstalling a Windows Agent

To uninstall an agent on a Windows system:

1. On the Windows system, open the Control Panel.

2. Go to the Programs and Features page (Control Panel > Programs > Uninstall a program).

3. Select the SiloAgent program from the list, and then click [Uninstall].

4. When the uninstallation process is complete, remove the agent configuration directory from the Windowssystem. The configuration directory can be found at:

Program Files\ScienceLogic\SiloAgent\conf

Installing an Agent from the Device Manager Page

If you are using a distributed SL1 system without the SL1 Extended Architecture (which includes Compute Nodes,Storage Nodes, and a Management Node), youmust use the Agent Installation page to install the agent. Youcan access the Agent Installation page from the Device Manager page.

TCP port 443 must be open between the Message Collector and the device on which an agent is installed.

NOTE: If you are using an SL1 system with the SL1 Extended Architecture, youmust use the Agents page(Devices > Agents) to install agents, including AIX and Solaris agents. AIX and Solaris agents are notavailable for non-SL1 Extended systems.

Installing an Agent from the Device Manager Page

Page 32: Monitoring with the SL1 Agent (Version 10.2.0)

Installing an Agent from the Device Manager Page

NOTE: If you are not using the SL1 Extended Architecture, the Agents page (Devices > Agents) will not beavailable in the SL1 user interface.

To install an agent from the Device Manager page, you first need to gather installation information from theAgent Installation page:

l For a Linux system, the Agent Installation page provides commands that must be executed on that system.

l For a Windows system, the Agent Installation page provides an executable file to run on the Windowssystem.

To gather the necessary commands and executable files to install an agent on a device:

1. Go to the Device Manager page (Device > Device Manager in the SL1 user interface, or Registry >Devices > Device Manager in the classic user interface).

2. Click [Actions] and select Download/Install Agent. The Agent Installation page appears:

3. Complete the following fields:

l Select an OS. Select the operating system running on the device on which you want to install theagent.

NOTE: If you require a FIPS-compliant version of the SL1 agent, select RedHat/CentOS 64-bit(OS Libs).

l Select an Organization. Select an organization from the list of possible organizations. The list oforganizations is dependent on your user account. If the agent discovers a new device, that device willbe associated with the organization you select here.

32

Page 33: Monitoring with the SL1 Agent (Version 10.2.0)

33

NOTE: If you are installing an agent on a device that has already been discovered, you mustselect the organization that is already aligned with the existing device.

l Select a Message Collector. Select the Message Collector to which the agent will send its collecteddata.

4. If you selected a Linux operating system in the Select an OS field, the Agent Installation page displays alist of commands to execute on the Linux system. Copy the commands for use during the installation on theLinux device.

5. If you selected a Windows operating system in the Select an OS field, the Agent Installation page displaysa Download Windows Agent link. Click the link and save the executable file for use during the installationon the Windows device.

TIP: If you are installing an agent on multiple devices that run the same operating system, are part of thesame organization, and connect to the same Message Collector, you can re-use the same commands orexecutable file on each of those devices.

Installing the Linux Agent from the Device Manager Page

To install an agent on a Linux system:

1. Log in to the Linux system via the console or SSH as a user that has sudo administrator permissions.

2. Execute the commands that you copied from the Agent Installation page in SL1. If the installation wassuccessful, the output will look similar to the following:

[em7admin@em7ao ~]$ sudo wget --no-check-certificatehttps://10.64.68.16/packages/initial/0/silo-agent-x86_64.rpm[sudo] password for em7admin:--2016-11-15 21:10:28-- https://10.64.68.16/packages/initial/0/silo-agent-x86_64.rpmConnecting to 10.64.68.16:443... connected.WARNING: cannot verify 10.64.68.16's certificate, issued by‘/C=US/ST=Silo/L=Reston/O=Silo/CN=10.64.68.16’:Self-signed certificate encountered.HTTP request sent, awaiting response... 200 OKLength: 2018317 (1.9M) [application/x-rpm]Saving to: ‘silo-agent-x86_64.rpm’100%[=======================================>] 2,018,317 --.-K/s in 0.01s2016-11-15 21:10:28 (169 MB/s) - ‘silo-agent-x86_64.rpm’ saved[2018317/2018317][em7admin@em7ao ~]$ sudo rpm -ihv silo-agent-x86_64.rpmPreparing... ################################# [100%]Updating / installing...1:scilogd-0.128-0 ################################# [100%]Created symlink from /etc/systemd/system/multi-user.target.wants/scilogd.service to /etc/systemd/system/scilogd.service.

Installing an Agent from the Device Manager Page

Page 34: Monitoring with the SL1 Agent (Version 10.2.0)

Installing an Agent from the Device Manager Page

NOTE: SL1 supports the use of proxy server connections when using the SL1 agent on Linux systems. To doso, open a command window on the target server and first configure curl to use a proxy server usingthe CURLOPT_PROXY option, and then to use a username and password combination for that proxyserver using the CURLOPT_PROXYUSERPWD option.

Updating an Agent on a Linux System

To update the agent on a Linux system:

1. Log in to the Linux system via the console or SSH as a user that has sudo administrator permissions.

2. Execute the first command that you copied from the Agent Installation page.

3. For RedHat-based Linux distributions, execute the following command:

sudo rpm -Uvh silo-agent-x86_64.rpm

4. For Ubuntu-based Linux distributions, execute the following command:

sudo dpkg -i silo-agent-x86_64.deb

Uninstall ing an Agent on a Linux System

To uninstall an agent on a Linux system:

1. Log in to the Linux system via the console or SSH as a user that has sudo administrator permissions.

2. For RedHat-based Linux distros, execute the following command:

rpm –e scilogd-0.128-0.[ARCH].rpm where [ARCH] = i386 or x86_64

3. For Ubuntu-based Linux distros, execute the following command:

dpkg –-purge silo-agent-[ARCH].deb where [ARCH] = i386 or x86_6

4. Remove the agent configuration directory from the Linux system. The configuration directory can be foundat:

/etc/scilog

34

Page 35: Monitoring with the SL1 Agent (Version 10.2.0)

35

Installing the Windows Agent from the Device Manager Page

To install an agent on a Windows system:

1. Copy the SiloAgent-install.exe file you downloaded from the Agent Installation page to the Windowssystem. You can go to the console of the Windows system or use a utility like WinSCP.

2. Run the following command as an Administrator:

SiloAgent-install.exe tenant=<organization_id> urlfront=<URL_for_your_SL1_system>

3. To verify that the installation was successful, open the Windows Task Manager or enter the TASKLISTcommand to view running processes. The SiloAgent process will be running on the Windows machine.

NOTE: SL1 supports the use of proxy server connections when using the SL1 agent on Windows systems. InWindows 10, go to Settings > Network & Internet > Proxy to set up a proxy; in older versions ofWindows, you can do this by clicking on the Control Panel and going to Internet Options> Connections > LAN Settings. If a proxy server requires authentication, you will need to provide ausername and password before traffic is allowed through the proxy server and to the Internet.

Uninstall ing the Agent on a Windows System

To uninstall an agent on a Windows system:

1. On the Windows system, open the Control Panel.

2. Go to the Programs and Features page (Control Panel > Programs > Uninstall a program).

3. Select the SiloAgent program from the list, and then click [Uninstall].

4. When the uninstallation process is complete, remove the agent configuration directory from the Windowssystem. The configuration directory can be found at:

Program Files\ScienceLogic\SiloAgent\conf

Configuring the SL1 Agent to Work with Phone Home Collectors

If you are using PhoneHome Message Collectors in a distributed architecture, you will need to configure thesettings for the Message Collector to prevent SL1 from incorrectly configuring the SL1 agent to use the loopbackIP address when it should be using the local IP address for the Message Collector.

If you do not perform the configuration steps below, when you try to install the agent from a Message Collectorwhen it is configured as a PhoneHome Collector, the URLs for both the download and the internal configurationof the agent reference the loopback address instead of the accessible public/private IP address of the MessageCollector.

Installing an Agent from the Device Manager Page

Page 36: Monitoring with the SL1 Agent (Version 10.2.0)

Installing an Agent from the Device Manager Page

NOTE: For more information about PhoneHome Collectors, see the "Configuring SL1 for PhoneHomeCommunication" chapter in the Installationmanual.

To configure the SL1 agent to work with a PhoneHome Collector:

1. Go to the Appliance Manager page (System > Settings > Appliances):

36

Page 37: Monitoring with the SL1 Agent (Version 10.2.0)

37

2. In the Message Collection Unit row, click the Agent Endpoint Configuration icon ( ) for the agent. TheAgent Endpoint Configurationmodal appears:

3. Make sure that the configuration information under the Appliance Details field is accurate.

4. Complete the following fields:

Installing an Agent from the Device Manager Page

Page 38: Monitoring with the SL1 Agent (Version 10.2.0)

Remotely Installing the SL1 Agent

l Download Link Visibility. Select whether you want to display the download link in theAgent Installationmodal page.

o If you select Enabled, the link to the Message Collector appears on the Agent Installationmodal, which is available on the Device Manager page (Devices > Device Manager orRegistry > Device > Device Manager in the classic user interface) when you click the[Actions] button and select Download/Install Agent:

o If you select Disabled, the link to the Message Collector does not appear in theAgent Installationmodal page.

l Endpoint Name for Download Link. This field contains the name of the Message Collector. If thename is not populated by default, you can enter the name.

l Hostname/IP for Agent Download. Make sure that this field contains the IP address for theMessage Collector.

l Hostname/IP for Data Collection. Enter the IP address for the Message Collector.

l Default CUG Alignment. Select the Collector Group aligned with the agent.

5. Click Save. The agent is now configured to work with a Message Collector when it is configured as a PhoneHome Collector.

Remotely Installing the SL1 Agent

This section covers how to configure remote installations of the Linux agent and the Windows agent.

NOTE: The following procedures are simply examples of how you could set up these installations, usingAnsible as the tool for the installation. You might use Ansible in a different way, or you might useanother tool altogether. These examples represent just one way to configure the remote installationof agents.

Remotely Installing the Linux Agent

This topic provides an example of how you can use Ansible to remotely install the SL1 Linux agent. Please refer tothe Ansible product documentation as needed.

To remotely install the Linux agent using Ansible:

38

Page 39: Monitoring with the SL1 Agent (Version 10.2.0)

39

1. Set up your Ansible control node, using the documentation at Installing Ansible.

2. Download the SL1 Linux agent installers from the Agents page (Devices > Agents > New Agent button) inSL1. For example:

l silo-agent-x86_64.rpm

l silo-agent-x86_64.deb

3. Create the hosts.yml file, using the following code as an example:

servers:hosts:

centos_host:ansible_host: 172.16.54.195ansible_user: root

debian_host:ansible_host: 10.2.16.125ansible_user: root

Remotely Installing the SL1 Agent

Page 40: Monitoring with the SL1 Agent (Version 10.2.0)

Remotely Installing the SL1 Agent

4. Create the playbook.yml file, using the following code as an example:

---- hosts: serversvars:

tenant: 0url_front: 10.2.16.49package_rpm: silo-agent-x86_64.rpmpackage_deb: silo-agent-x86_64.deb

tasks:

- name: check if agent existsstat:

path: /usr/bin/scilogdregister: agent_binary

- name: copy rpm installercopy:

src: "{{ package_rpm }}"dest: ~/{{ package_rpm }}

when: (agent_binary.stat.exists == False and ansible_facts['os_family'] =='RedHat')

- name: copy deb installercopy:

src: "{{ package_deb }}"dest: ~/{{ package_deb }}

when: (agent_binary.stat.exists == False and ansible_facts['os_family'] =='Debian')

- name: install rpm agentshell: TENANT={{ tenant }} URLFRONT={{ url_front }} rpm -ihv ~/{{ package_

rpm }}args:

creates: /usr/bin/scilogdwhen: (agent_binary.stat.exists == False and ansible_facts['os_family'] ==

'RedHat')

- name: install deb agentshell: TENANT={{ tenant }} URLFRONT={{ url_front }} dpkg -i ~/{{ package_

deb }}args:

creates: /usr/bin/scilogdwhen: (agent_binary.stat.exists == False and ansible_facts['os_family'] ==

'Debian')

- name: remove rpm installerfile:

path: ~/{{ package_rpm }}state: absent

when: (agent_binary.stat.exists == False and ansible_facts['os_family'] =='RedHat')

- name: remove deb installerfile:

40

Page 41: Monitoring with the SL1 Agent (Version 10.2.0)

41

path: ~/{{ package_deb }}state: absent

when: (agent_binary.stat.exists == False and ansible_facts['os_family'] =='Debian')

Remotely Installing the SL1 Agent

Page 42: Monitoring with the SL1 Agent (Version 10.2.0)

Remotely Installing the SL1 Agent

5. Run the playbook by executing the following command:

ansible-playbook playbook.yml -i hosts.yml --ask-pass

NOTE: This example uses the --ask-pass parameter so that you are prompted for a singlepassword to use with all remote computers.

The following code is an example of the output:

$ ansible-playbook playbook.yml -i hosts.yml --ask-passSSH password:

PLAY [servers]**************************************************************************************************************

TASK [Gathering Facts]******************************************************************************************************ok: [centos_host]ok: [debian_host]

TASK [check if agent exists]************************************************************************************************ok: [centos_host]ok: [debian_host]

TASK [copy rpm installer]***************************************************************************************************skipping: [debian_host]changed: [centos_host]

TASK [copy deb installer]***************************************************************************************************skipping: [centos_host]changed: [debian_host]

TASK [install rpm agent]****************************************************************************************************skipping: [debian_host]changed: [centos_host]

TASK [install deb agent]****************************************************************************************************skipping: [centos_host]changed: [debian_host]

TASK [remove rpm installer]

42

Page 43: Monitoring with the SL1 Agent (Version 10.2.0)

43

*************************************************************************************************skipping: [debian_host]changed: [centos_host]

TASK [remove deb installer]*************************************************************************************************skipping: [centos_host]changed: [debian_host]

PLAY RECAP******************************************************************************************************************centos_host : ok=5 changed=3 unreachable=0 failed=0skipped=3 rescued=0 ignored=0debian_host : ok=5 changed=3 unreachable=0 failed=0skipped=3 rescued=0 ignored=0

Remotely Installing the SL1 Windows Agent

This topic provides examples of how you can use the PowerShell console or Ansible to remotely install the SL1Windows agent. Please refer to the PowerShell or Ansible product documentation as needed.

To remotely install the Windows agent using InstallSoftwareRemotely.ps1:

1. Download the Windows Agent installer from the Agents page (Devices > Agents > New Agent button) inSL1 and add the installer to your domain-controller n its own directory. For example, C:\files\SiloAgent-install.exe.

2. Download the third-party PowerShell script to your domain-controller.

3. Create a comma-separated value (CSV) file with the names of the computers where you want to install theagent. For example: C:\computers.csv, such as:

rstlsw12t16ls01rstlsw12t16ex01computer3computer99

Remotely Installing the SL1 Agent

Page 44: Monitoring with the SL1 Agent (Version 10.2.0)

Remotely Installing the SL1 Agent

4. Run the following command from a PowerShell console:

.\InstallSoftwareRemotely.ps1 -CSV C:\computers.csv -AppPath 'C:\foo\SiloAgent-install.exe' -AppArgs "Tenant=<organization_id> urlFront=<streamer_URL>"

where:

l -CSV uses the full path to the .csv file you created in step 3, above.

l -AppPath uses the full path to the installer in step 1, above. Make sure the file is in its own directory.

l -AppArgs uses the Tenant and urlFront values shown on the Windows agent install instructionspage in SL1.

l You can optionally use the -Credential option to be prompted for a different username andpassword to access all remote machines. Otherwise the script will use your current login account toaccess all remote machines.

The following code is an example of the output:

PS C:\> .\InstallSoftwareRemotely.ps1 -CSV C:\computers.csv -AppPath'C:\foo\SiloAgent-install.exe'-AppArgs "Tenant=0 URLFront=streamer-shared-e2e-testing.int.sciencelogic.com"Start remote installation on 2020-56-20 09:56:17No credential specified. Using logon account-----------------------------------------------------------------Attempt 1 of 5-----------------------------------------------------------------COMPUTER RSTLSW12T16LS01 (1 of 2)Coping C:\foo to \\rstlsw12t16ls01\C$\tempInstalling SiloAgent-install.exe on RSTLSW12T16LS01Executing C:\temp\foo\SiloAgent-install.exe Tenant=0 URLFront=streamer-shared-e2e-testing.int.sciencelogic.comDeleting C:\temp\fooSiloAgent-install.exe installed successfully.COMPUTER RSTLSW12T16EX01 (2 of 2)Coping C:\foo to \\rstlsw12t16ex01\C$\tempInstalling SiloAgent-install.exe on RSTLSW12T16EX01Executing C:\temp\foo\SiloAgent-install.exe Tenant=0 URLFront=streamer-shared-e2e-testing.int.sciencelogic.comDeleting C:\temp\fooSiloAgent-install.exe installed successfully.-----------------------------------------------------------------100% Success installing SiloAgent-install.exe on 2 of 2 computers:rstlsw12t16ls01 rstlsw12t16ex01

To remotely install the Windows agent using Ansible:

1. Set up your Ansible control node. Refer to Ansible documentation for installing on other operatingsystems: Installing Ansible with pip.

2. Update your Ansible control node to manage Windows servers by running the following command:

sudo pip3 install pywinrm

44

Page 45: Monitoring with the SL1 Agent (Version 10.2.0)

45

3. Ensure that remote machines are set up with PowerShell 3 or later:

https://github.com/jborean93/ansible-windows/blob/master/scripts/Upgrade-PowerShell.ps1

4. Ensure that remote machines are setup for Windows Remote Management:

https://github.com/ansible/ansible/blob/devel/examples/scripts/ConfigureRemotingForAnsible.ps1

5. Download the SL1 Windows agent installer, SiloAgent-install.exe, from the Agents page (Devices >Agents > New Agent button) in SL1.

6. Create the hosts.yml file, using the following code as an example:

group1:hosts:

win2008r2:ansible_host: 10.2.16.41ansible_user: Administrator

win2012:ansible_host: 10.2.16.42ansible_user: Administrator

win2012r2:ansible_host: 10.2.16.40ansible_user: Administrator

group2:hosts:

dc01:ansible_host: 10.2.16.60ansible_user: administrator@T16TESTDOMAIN

ls01:ansible_host: 10.2.16.62ansible_user: administrator@T16TESTDOMAIN

es01:ansible_user: administrator@T16TESTDOMAINansible_host: 10.2.16.64

7. Create themy_playbook.yml file, using the following code as an example:

---- hosts: group1gather_facts: notasks:- name: Ping Windows Hosts

win_ping:- name: Copy agent installer

win_copy:src: ./SiloAgent-install.exedest: C:\Temp\SiloAgent-install.exe

- name: Exec Installerwin_command: C:\Temp\SiloAgent-install.exe Tenant=yyy URLFront=zzz

- name: Remove agent installerwin_file:

path: C:\Temp\SiloAgent-install.exestate: absent

Remotely Installing the SL1 Agent

Page 46: Monitoring with the SL1 Agent (Version 10.2.0)

Remotely Installing the SL1 Agent

8. Run the playbook by executing the following command:

ansible-playbook playbook.yml -i hosts.yml --ask-pass

NOTE: This example uses the --ask-pass parameter so that you are prompted for a singlepassword to use with all remote computers. Alternately, you could include the password inhosts.yml or ./group_vars/all.yml.

The following code is an example of the output:

[em7admin@jkay88 demo_ansible]$ ansible-playbook my_playbook.yml -i hosts.yml --ask-passSSH password:

PLAY [group1]************************************************************************************************************************************************************************************

TASK [Ping Windows Hosts]************************************************************************************************************************************************************************ok: [win2012r2]ok: [win2012]ok: [win2008r2]

TASK [Copy agent installer]**********************************************************************************************************************************************************************changed: [win2012]changed: [win2012r2]changed: [win2008r2]

TASK [Exec Installer]****************************************************************************************************************************************************************************changed: [win2012]changed: [win2012r2]changed: [win2008r2]

PLAY RECAP***************************************************************************************************************************************************************************************win2008r2 : ok=3 changed=2 unreachable=0 failed=0skipped=0 rescued=0 ignored=0win2012 : ok=3 changed=2 unreachable=0 failed=0skipped=0 rescued=0 ignored=0win2012r2 : ok=3 changed=2 unreachable=0 failed=0skipped=0 rescued=0 ignored=0

46

Page 47: Monitoring with the SL1 Agent (Version 10.2.0)

Chapter

3Configuring the SL1 Agent

Overview

This chapter covers how to configure the agent monitoring based on the SL1 architecture you are using. Thischapter also describes additional agent configurations, such as enabling extensible collection with an SL1 agent.

Use the following menu options to navigate the SL1 user interface:

l To view a pop-out list of menu options, click the menu icon ( ).

l To view a page containing all the menu options, click the Advanced menu icon ( ).

This chapter includes the following topics:

Configuring Agent Monitoring Based on SL1 Architecture 48

Configuring Agent Settings from the Device Investigator Page 49

Configuring Extensible Collection 52

Configuring an SL1 Agent on the Device Manager Page 54

47

Page 48: Monitoring with the SL1 Agent (Version 10.2.0)

48

Configuring Agent Monitoring Based on SL1 Architecture

Based on your SL1 architecture, you will have access to different workflows for configuring the monitoring settingson an SL1 agent:

l If the SL1 agent is used in an SL1 Extended Architecture (which includes Compute Nodes, Storage Nodes,and a Management node), use the following configuration workflows:

o To configure basic settings related to how the agent collects data, see Configuring Agent Settingsfrom the Device Investigator Page.

o To configure log file monitoring and Windows event log monitoring with the agent, seeMonitoringDevice Logs Using the Agent.

o To configure and view data collected by the agent, see the relevant tabs on the Device Investigatorpage, including the following tabs:

n [Settings]

n [Interfaces]

n [Processes]

n [Services]

n [Collections]

For more information about these tabs, see the "Using the Device Investigator" chapter in theDevice Managementmanual.

l If the SL1 agent is used in a distributed SL1 architecture (where the SL1 Agent collects and transfers datato a Message Collector), use the following configuration workflows:

o To configure basic settings related to how the agent collects data, see Configuring an SL1 Agenton the Device Manager Page.

o To configure log file monitoring and Windows event log monitoring with the agent, seeMonitoringDevice Logs Using the Agent.

o To configure and view data collected by the agent, see the relevant tabs on the Device Investigatorpage, including the following tabs:

n [Settings]

n [Interfaces]

n [Processes]

n [Services]

n [Collections]

For more information about these tabs, see the "Using the Device Investigator" chapter in theDevice Managementmanual.

Configuring Agent Monitoring Based on SL1 Architecture

Page 49: Monitoring with the SL1 Agent (Version 10.2.0)

Configuring Agent Settings from the Device Investigator Page

o To configure system vitals monitoring with the agent, see the "Monitoring Vitals Using an Agent"chapter of the Device Infrastructure Healthmanual.

o To configure port monitoring with the agent, see the "Monitoring Ports" chapter of the DeviceInfrastructure Healthmanual.

Configuring Agent Settings from the Device Investigator Page

If a device does not have an agent running on it, the [Settings] tab of the Device Investigator page for thedevice will display the Collection section at top left. However, if the device has an agent running on it, theAgents section appears above the Collection section:

In the Agents section, you can configure the disk space, excludes, includes, and other settings related to how youwant the agent to collect data.

NOTE: The Agents section appears on the [Settings] tab only when you are using the SL1 ExtendedArchitecture. If your version of SL1 does not have an Agents section, see Configuring an SL1Agent on the Device Manager Page.

49

Page 50: Monitoring with the SL1 Agent (Version 10.2.0)

50

To configure agent settings:

1. From the Devices page, select the device with the agent that you want to configure. TheDevice Investigator page appears.

TIP: To quickly find all devices on the Devices page that have agents installed on them, type "agent" inthe Search field and select the "ANY: agent" search.

2. Click the [Settings] tab and click [Edit.]

3. In the Agent section, complete the following fields, as needed:

Configuring Agent Settings from the Device Investigator Page

Page 51: Monitoring with the SL1 Agent (Version 10.2.0)

Configuring Agent Settings from the Device Investigator Page

l Disk Space. Specify the amount of disk space in MB that the agent can use to store data. If theagent loses connectivity to SL1, this disk space will be used to store collected data until theconnection to SL1 is restored. When connectivity is re-established, the agent uploads all of its storeddata.

l Excludes. Type a list of processes and directories, separated by semi-colons, that you do not wantthe agent to monitor.

l Includes. Type a list of processes and directories, separated by semi-colons, that you want theagent to monitor. This field ensures that specific processes are monitored.

NOTE: If a process or directory is included in both the Excludes field and the Includes field,the item in the Includes field will override the item in the Excludes field.

l Collect File Information. Select this option if you want the agent to report the names of filesaccessed by each monitored process.

l Collect Named Pipe Information. Select this option if you want the agent to collect named pipeinformation.

l Collect Socket Information. Select this option if you want the agent to collect socket information.

l Collect Thread Information. Select this option if you want the agent to collect thread information.

l Collect Non-Intercepted Processes. Select this option if you want the agent to collect limitedinformation for processes that do not contain the agent library.

l Processes Aggregation. Specify how you want the agent to collect limited information forprocesses that do not have the agent library in them, and how to aggregate short-lived processes.Your options include the following:

o All: Aggregate every short-lived process into its parent.

o None: Do not aggregate any short-lived process.

o Without Sockets: Aggregate short-lived processes unless those processes have sockets.

l Upload Interval. Specify how often the agent should upload data. Your options include thefollowing:

l 20 Seconds. Upload a data snapshot every 20 seconds. This is the default setting, and it isthe same interval used with the distributed SL1 architecture.

l 60 Seconds. Upload a data summary every 60 seconds. This option uses an improved dataformat that requires fewer SL1 resources. The SL1 agent continues to internally collect andpoll data every 20 seconds, but the agent summarizes and uploads that data every 60seconds. There is no data loss even though the data is uploaded less frequently.

NOTE: The Upload Interval option is currently available only with version 132 orlater of the Windows SL1 agent. The Linux agent only uploads snapshots, notsummaries.

51

Page 52: Monitoring with the SL1 Agent (Version 10.2.0)

52

4. Click the [Save] button to save your configuration settings for the agent.

TIP: You can view and configure additional agent settings from the Device Investigator page for the deviceon which the agent is installed, including the following tabs: [Interfaces], [Processes], [Services], and[Collections]. For more information about these tabs, see the Using the Device Investigator chapter inthe Device Managementmanual.

Configuring Extensible Collection

You can use the SL1 agent for "extensible collection", where you align the SL1 agent with Dynamic Applications togather metrics and attributes from other infrastructures and applications.

Dynamic Applications are customizable policies, created for a specific vendor and a specific type of device orsystem, that tell SL1:

l What data to collect from devices

l How to present the data that has been collected

l When to generate alerts and events based on the data that has been collected

SL1 includes Dynamic Applications for the most common hardware and software. You can customize thesedefault Dynamic Applications to best work in your environment. You can also create custom DynamicApplications.

You can align the agent with PowerShell Dynamic Applications (for Windows agents) and JMX DynamicApplications (for Linux agents).

TIP: Before starting the configuration process, make sure that the Account Type is set to Local for thecredential you are using to align the Dynamic Application to the agent. Do not use the Device SNMPcredential.

To enable extensible collection:

1. On the Devices page, search for the device with the agent installed on it.

TIP: To quickly find all devices that have agents installed on them, type "agent" in the Search field onthe Devices page, and select the "ANY: agent" search.

2. Select the device. The Device Investigator page appears.

Configuring Extensible Collection

Page 53: Monitoring with the SL1 Agent (Version 10.2.0)

Configuring Extensible Collection

3. Click the [Collections] tab, click [Edit], and click [Align Dynamic App]. The Align Dynamic Applicationwindow appears:

4. Click Choose Dynamic Application and search for the Dynamic Application you want to align.

5. Select the Dynamic Application and click [Select]. The Align Dynamic Application window appearsagain.

6. Click Choose Credential to search for a credential for the agent. The Choose Credential page appears.

NOTE: The Account Type for the credential must be Local for the agent. Do not use the SNMPcredential.

7. Select the credential and click [Select]. The Align Dynamic Application window appears again.

53

Page 54: Monitoring with the SL1 Agent (Version 10.2.0)

54

8. Click [Align Dynamic App]. The [Collections] tab appears, and a pop-up message displays when theDynamic Application is aligned.

9. Click [Save].

10. To view the metrics from that newly aligned Dynamic Application, click the [Investigator] tab.

11. In the Add a metric field, select a metric from the new Dynamic Application to view its data in the right-hand pane.

NOTE: The metrics from the new Dynamic Application might take a minute or two to display in the listof metrics. Click [Refresh] in your browser if needed.

Configuring an SL1 Agent on the Device Manager Page

NOTE: To configure agent settings for an agent, you must first add the SL Agent column to the DeviceManager page. For more information about adding the SL Agent column, see Adding theSL Agent Column to the Device Manager Page.

NOTE: If you are using the SL1 Extended Architecture, you can access these settings from the [Settings] tabof the Device Investigator page for the device on which the agent is installed. For moreinformation, see Configuring Agent Settings from the Device Investigator Page.

You can control how an agent runs on a device by configuring the following agent settings on the DeviceManager page (Devices > Device Manager or Registry > Devices > Device Manager):

l Disk Space. Controls the amount of disk space that the agent can use to store data. If an agent losesconnectivity to SL1, this disk space will be used to store collected data until the connection to SL1 is restored.

l Data Directory. Defines the directory in which the agent will store temporary data.

Configuring an SL1 Agent on the Device Manager Page

Page 55: Monitoring with the SL1 Agent (Version 10.2.0)

Configuring an SL1 Agent on the Device Manager Page

l Excludes. Defines the list of processes and directories to explicitly exclude from monitoring by the agent.

l Includes. Defines the list of processes and directories that must be explicitly monitored by the agent. Use theIncludes field to ensure that specific processes are monitored.

NOTE: If a process or directory is included in both the Excludes field and the Includes field, thatprocess or directory will be monitored by the agent.

Adding the "SL Agent" Column to the Device Manager Page

The SL Agent column allows you to access the configuration settings for the agent on a device. For moreinformation about agent configuration settings, see Configuring Agent Settings on a Device. By default, the SLAgent column is not displayed in the Device Manager page.

To add the SL Agent column to the Device Manager page:

1. Go to the Device Manager page (Devices > Device Manager or Registry > Devices > Device Manager inthe classic user interface).

2. Click [Actions], and then select Device Manager Preferences. The Edit Device Manager Preferencesmodal page appears:

3. In the Device Manager Columns field, control-click Agent.

4. Click [Save].

55

Page 56: Monitoring with the SL1 Agent (Version 10.2.0)

56

Configuring Agent Settings on a Device

To configure agent settings, you must first add the SL Agent column to the Device Manager page. For moreinformation about adding the SL Agent column, see Adding the SL Agent Column to the Device ManagerPage.

To configure agent settings on a device:

1. Go to the Device Manager page (Devices > Device Manager or Registry > Devices > Device Manager).

2. Find the device for which you want to edit agent settings. In the SL Agent column, click the gear icon ( )for the device. The Agent Configuration page appears:

3. Supply values in the following fields:

l Disk Space. Enter the amount of disk space that the agent can use to store data. If the agent losesconnectivity to SL1, this disk space will be used to store collected data until the connection to SL1 isrestored.

l Data Directory. Enter the directory in which the agent will store temporary data.

l Excludes. Enter a semi-colon delimited list of processes and directories to explicitly exclude frommonitoring by the agent.

l Includes. Enter a semi-colon delimited list of processes and directories that must be monitored by theagent. Use the Includes field to ensure that specific processes are monitored.

Configuring an SL1 Agent on the Device Manager Page

Page 57: Monitoring with the SL1 Agent (Version 10.2.0)

Configuring an SL1 Agent on the Device Manager Page

NOTE: If a process or directory is included in both the Excludes field and the Includes field,that process or directory will be monitored by the agent.

4. Click [Save].

57

Page 58: Monitoring with the SL1 Agent (Version 10.2.0)

Chapter

4Monitoring Logs Using the SL1 Agent

Overview

This chapter describes how to use the SL1 Agent to monitor logs with Log File Monitoring policies. SL1 supportsmultiple methods for ingesting log data, which you can use to generate events.

Use the following menu options to navigate the SL1 user interface:

l To view a pop-out list of menu options, click the menu icon ( ).

l To view a page containing all the menu options, click the Advanced menu icon ( ).

This chapter includes the following topics:

What is a Log File Monitoring Policy? 59

Viewing the List of Log File Monitoring Policies 59

Creating a Log File Monitoring Policy 61

Aligning a Log File Monitoring Policy to Devices 63

Editing a Log File Monitoring Policy 66

Deleting Log File Monitoring Policies 67

Viewing the List of Log File Monitoring Policies and Aligned Devices 67

Creating an Event Policy for Agent Logs 68

Creating an Event Policy for Agent Logs in the Classic User Interface 73

58

Page 59: Monitoring with the SL1 Agent (Version 10.2.0)

59

What is a Log File Monitoring Policy?

A Log File Monitoring policy specifies:

l A file or Windows log on the host device that an agent will monitor

l The logs from the file or Windows log that an agent will send to SL1

You can create, edit, and delete Log File Monitoring policies from the Log File Monitoring Policies page(System > Manage > Log File Monitoring Policies). After creating a Log File Monitoring policy, you must alignthe policy to one or more devices either from the Log File Monitoring page or by using a Device Template.

You can view logs collected by the SL1 agent on the Logs panel on the Device Investigator page for the deviceon which the agent is installed. The same logs also appear on the [Logs] tab in the Device Properties andDevice Summary pages for that device. You can define event policies that specify how logs collected by anagent will trigger events.

Log File Monitoring policies can be included in a PowerPack. For information about including a Log FileMonitoring Policy in a PowerPack, see the PowerPacksmanual.

Viewing the List of Log File Monitoring Policies

The Log File Monitoring Policies page (System > Manage > Log File Monitoring Policies) displays a list of allLog File Monitoring policies. From this page, you can also create, edit, and delete Log File Monitoring policies.

What is a Log File Monitoring Policy?

Page 60: Monitoring with the SL1 Agent (Version 10.2.0)

Viewing the List of Log File Monitoring Policies

TIP: To sort the list of Log File Monitoring policies, click on a column heading. The list will be sorted by thecolumn value, in ascending order. To sort by descending order, click the column heading again. TheLast Edited column sorts by descending order on the first click; to sort by ascending order, click thecolumn heading again.

For each Log File Monitoring Policy, the page displays:

l Name. Name of the Log File Monitoring policy.

l Policy ID. Unique numeric ID, automatically assigned by SL1 to each Log File Monitoring policy.

l Source Type. The source of the logs on the monitored device. Possible values are:

o File. The agent will monitor a file on the file system of the device(s).

o Event Log. The agent will monitor the Windows log on the device(s).

l Source. The full path of the log file or the name of the Windows log that the agent will monitor.

l Filter. The regular expression that the agent uses to determine whether a log message is sent to SL1.

l Subscribers. The number of devices with which the policy is aligned.

l Edited By. SL1 user who created or last edited the Log File Monitoring policy.

l Last Edited. Date and time the Log File Monitoring policy was created or last edited.

Filtering the List of Log File Monitoring Policies

To filter the list of credentials in the Log File Monitoring Policies page, use the search fields at the top of eachcolumn. The search fields are find-as-you-type filters; as you type, the page is filtered to match the text in thesearch field, including partial matches. Text matches are not case-sensitive. Additionally, you can use thefollowing special characters in each filter:

l , (comma). Specifies an "or" operation. For example:

"dell, micro" would match all values that contain the string "dell" OR the string "micro".

l & (ampersand). Specifies an "and" operation. For example:

"dell & micro" would match all values that contain the string "dell" AND the string "micro".

l ! (exclamation mark). Specifies a "not" operation. For example:

"!dell" would match all values that do not contain the string "dell".

l ^ (caret mark). Specifies "starts with." For example:

"^micro" would match all strings that start with "micro", like "microsoft".

"^" will include all rows that have a value in the column.

"!^" will include all rows that have no value in the column.

60

Page 61: Monitoring with the SL1 Agent (Version 10.2.0)

61

l $ (dollar sign). Specifies "ends with." For example:

"$ware" would match all strings that end with "ware", like "VMware".

"$" will include all rows that have a value in the column.

"!$" will include all rows that have no value in the column.

l min-max. Matches numeric values only. Specifies any value between the minimum value and the maximumvalue, including the minimum and the maximum. For example:

"1-5" would match 1, 2, 3, 4, and 5.

l - (dash). Matches numeric values only. A "half open" range. Specifies values including the minimum andgreater or including the maximum and lesser. For example:

"1-" matches 1 and greater, so it would match 1, 2, 6, 345, etc.

"-5" matches 5 and less, so it would match 5, 3, 1, 0, etc.

l > (greater than). Matches numeric values only. Specifies any value "greater than." For example:

">7" would match all values greater than 7.

l < (less than). Matches numeric values only. Specifies any value "less than." For example:

"<12" would match all values less than 12.

l >= (greater than or equal to). Matches numeric values only. Specifies any value "greater than or equal to."For example:

"=>7" would match all values 7 and greater.

l <= (less than or equal to). Matches numeric values only. Specifies any value "less than or equal to." Forexample:

"=<12" would match all values 12 and less.

l = (equal). Matches numeric values only. For numeric values, allows you to match a negative value. Forexample:

"=-5 " would match "-5" instead of being evaluated as the "half open range" as described above.

Creating a Log File Monitoring Policy

To create a Log File Monitoring policy:

1. Go to the Log File Monitoring Policies page (System > Manage > Log File Monitoring Policies).

Creating a Log File Monitoring Policy

Page 62: Monitoring with the SL1 Agent (Version 10.2.0)

Creating a Log File Monitoring Policy

2. Click [Create]. The Log Monitoring Policymodal appears:

3. Supply values in the following fields:

l Name. Enter a name for the policy.

l Type. Select the source of the logs on the monitored device. Choices are:

o File. The agent will monitor a file on the file system of the device(s).

o Event Log. The agent will monitor the Windows log on the device(s).

l File Path. If you selected File in the Type field, this field is displayed. Enter the full path of the file tomonitor. You can use a * to match multiple files, such as /var/log/em7/*.log.

l Source. If you selected Event Log in the Type field, this field is displayed. Select the Windows log tomonitor. Choices are:

o application

o system

o security

l Limit. The maximum log messages the agent sends to SL1 per minute. If the number of matching logsexceeds this value, the agent will stop sending logs to the platform for the remainder of the minute.The limit resets at the beginning of the next minute.For example, suppose you set this field to 10,000. Suppose the agent monitors a device that has30,000 log messages. The agent will retrieve 10,000 logs and then wait until the beginning of thenext minute. The agent will then retrieve the next 10,000 logs and then wait until the beginning of thenext minute. The agent will continue to retrieve 10,000 logs per minute until it has retrieved all thelogs from the device.

62

Page 63: Monitoring with the SL1 Agent (Version 10.2.0)

63

l Filter. Specify a regular expression that will be used to evaluate the log messages in the specified fileor Windows log. If a log message matches this regular expression, the agent will send that logmessage to SL1. If a log message does not match this regular expression, the agent will not send thatlog message to SL1.

NOTE: For Windows event logs, the SL1 Agent adds the Event ID to the value in theMessageportion of the Windows log before applying the value in the Filter field. The agent doesnot apply the value in the Filter field to the Instance ID or any other property of aWindows event log entry.

4. Click [Save].

5. Before you can use this Log File Monitoring policy, you will need to align the policy with one or moredevices. For more information, see Aligning a Log File Monitoring Policy to Devices.

Aligning a Log File Monitoring Policy to Devices

You can align Log File Monitoring policies to devices either from the Log File Monitoring page or by using aDevice Template.

This section describes how to align a Log File Monitoring policy from the Log File Monitoring page. It alsodescribes how to use a one-off Device Template to align a Log File Monitoring policy. For more information onDevice Templates, including the other methods you can use to create, save, and apply Device Templates, see theDevice Groups and Device Templatesmanual.

To align Log File Monitoring policies to one or more devices from the Log File Monitoring page:

1. Go to the Log File Monitoring page (Registry > Monitors > Logs).

Aligning a Log File Monitoring Policy to Devices

Page 64: Monitoring with the SL1 Agent (Version 10.2.0)

Aligning a Log File Monitoring Policy to Devices

2. Click [Create]. The Log File Monitor modal appears:

3. In the Log File Monitor modal, supply values in the following fields:

l Device. Select a device to align with the Log File Monitoring policy.

l Log Policy. Select the Log File Monitoring policy to align with the selected device. Only policies thatare appropriate for the selected device will appear. For example, if you chose a Linux device in theDevice field, the Log Policy field will not show policies of the Event Log type.

4. If desired, click the names of the following fields to enable and edit them. These fields allow you to overridesettings of the policy you selected in the Log Policy field for the device selected in the Device field:

l File Path. Enter the full file path or the file name to monitor. This field appears only if the type of thepolicy is File.

l Limit. The maximum log messages the agent sends to SL1 per minute. If the number of matching logsexceeds this value, the agent will stop sending logs to the platform for the remainder of the minute.The limit resets at the beginning of the next minute. For example, suppose you set this field to 10,000.Suppose the agent monitors a device that has 30,000 log messages. The agent will retrieve 10,000logs and then wait until the beginning of the next minute. The agent will then retrieve the next 10,000logs and then wait until the beginning of the next minute. The agent will continue to retrieve 10,000logs per minute until it has retrieved all the logs from the device.

64

Page 65: Monitoring with the SL1 Agent (Version 10.2.0)

65

l File. Specify a regular expression that will be used to evaluate the log messages in the specified file orWindows log. If and only if a log message matches this regular expression, the agent will send the logmessage to SL1.

5. Click [Save].

To align Log File Monitoring policies to one or more devices using a Device Template:

1. Go to the Device Manager page (Devices > Device Manager).

2. Select the checkboxes for the devices with which you want to align Log File Monitoring policies.

3. In the Select Action drop-down list, selectMODIFY by Template.

4. Click [Go]. The Device Template Editormodal appears:

5. Click the [Logs] tab.

6. Click the Add New Log Policy Sub-Template icon ( ).

7. Supply values in the following fields:

l Align Log Monitoring Policy With. Select the devices to which the Log File Monitoring policy will beapplied.

l Log Monitoring Policy. Select the Log File Monitoring policy you want to align with the selecteddevices.

Aligning a Log File Monitoring Policy to Devices

Page 66: Monitoring with the SL1 Agent (Version 10.2.0)

Editing a Log File Monitoring Policy

8. Optionally, you can override one or more settings from the Log File Monitoring policy specifically for theselected devices. To do this, click the field label for each setting you want to override to enable the fieldsand supply a value in those fields. For a description of each field, see the Creating a Log File MonitoringPolicy section.

9. Repeat steps 6 and 7 for each Log File Monitoring policy you want to align with the devices you selected instep 2.

10. If you want to save this Device Template for future use, select the Save When Applied & Confirmedcheckbox and enter a name for the Device Template in the Template Name field.

11. Click [Apply]. The Setting Confirmation page is displayed.

12. Click [Confirm]. The aligned Log File Monitoring policy will appear on the Log File Monitoring page(Registry > Monitors > Logs).

Unaligning Log File Monitoring Policies from Devices

To delete Log File Monitoring Policies, you must first unalign the policy from any devices. You can unalign a LogFile Monitoring policy by from the Log File Monitoring page.

To unalign devices from a Log File Monitoring policy:

1. Go to the Log File Monitoring page (Registry > Monitors > Logs)

2. Select the checkboxes for the devices from which the policy must be unaligned.

3. In the Select Action drop-down menu, choose Delete Log File Monitors.

NOTE: This action does not delete the Log File Monitoring policy.

4. Click [Go] to unalign the Log File Monitoring policy from the devices.

Editing a Log File Monitoring Policy

To edit a Log File Monitoring policy:

1. Go to the Log File Monitoring Policies page (System > Manage > Log File Monitoring Policies).

2. Click the wrench icon ( ) for the Log File Monitoring Policy you want to edit. The Log Monitoring Policymodal appears.

3. Edit the value in one or more fields. For a description of each field, see the Creating a Log File MonitoringPolicy section.

4. Click [Save].

66

Page 67: Monitoring with the SL1 Agent (Version 10.2.0)

67

Deleting Log File Monitoring Policies

NOTE: Before you delete a Log File Monitoring Policy, you must unalign that policy from all devices. SeeUnaligning Log File Monitoring Policies for more information.

To delete one or more Log File Monitoring policies:

1. Go to the Log File Monitoring Policies page (System > Manage > Log File Monitoring Policies).

2. Select the checkboxes for the Log File Monitoring Policies you want to delete.

3. In the Select Action drop-down list, select DELETE Log FIle Monitoring Policies.

4. Click [Go].

Viewing the List of Log File Monitoring Policies and AlignedDevices

The Log File Monitoring page (Registry > Monitors > Logs) displays a list of existing relationships betweendevices and Log File Monitoring policies. From the Log File Monitoring page, you can also align and unaligndevices and Log File Monitoring policies.

For each aligned Log File Monitoring policy and device, the page displays:

l Name. The name of the Log File Monitoring policy.

l Device Name. The name of the device aligned to the Log File Monitoring policy.

l ID. The unique numeric ID of the Log File Monitoring policy. The ID is automatically assigned by SL1.

Deleting Log File Monitoring Policies

Page 68: Monitoring with the SL1 Agent (Version 10.2.0)

Creating an Event Policy for Agent Logs

l Source Type. The source of the logs in the monitored device. The possible values are:

o File. The agent monitors a file on the file system of the device. Usually, this is used to monitor Linuxlog files.

o Event Log. The agent monitors to Windows log on the device.

l Source. The full path of the log file or the name of the Windows log that the agent monitors.

l Filter. The regular expression the agent uses to determine if a log should be sent to SL1.

l Limit. The maximum log messages the agent sends to SL1 per minute. If the number of matching logsexceeds this value, the agent will stop sending logs to the platform for the remainder of the minute. The limitresets at the beginning of the next minute. For example, suppose you set this field to 10,000. Suppose theagent monitors a device that has 30,000 log messages. The agent will retrieve 10,000 logs and then waituntil the beginning of the next minute. The agent will then retrieve the next 10,000 logs and then wait untilthe beginning of the next minute. The agent will continue to retrieve 10,000 logs per minute until it hasretrieved all the logs from the device.

l Edited By. The user who created or last edited the alignment between the device and Log File Monitoringpolicy.

l Last Edited. The date and time the alignment between the device and Log File Monitoring policy wascreated or last edited.

Filtering the List of Log File Monitoring Policies and Aligned Devices

You can filter the list of Log File Monitoring policies and aligned devices on the Log File Monitoring page usingthe search fields at the top of each column. When you type in each search field, the list of results on the page isautomatically updated to match the text, including partial matches.

You can use special characters in each search field to filter. Fore more information about filtering using specialcharacters, see the Filtering the List of Log File Monitoring Policies section.

Creating an Event Policy for Agent Logs

To trigger events based on log messages collected by the agent, you must create an event policy on theEvent Policies page, and then associate that event policy with a Log File Monitoring policy.

NOTE: If you are using the classic user interface for SL1, see Creating an Event Policy for Agent Logs inthe Classic User Interface.

To create an event policy that triggers based on log data collected by the agent:

1. Go to Event Policies page (Events > Event Policies).

68

Page 69: Monitoring with the SL1 Agent (Version 10.2.0)

69

2. On the Event Policies page, click the [Create Event Policy] button. The Event Policy Editor pageappears:

3. On the Event Policy Editor page and set of tabs, you can define a new event. The Event Policy Editorpage contains the following fields and tabs:

l Policy Name. At the top left of the [Policy Description] tab, type a name for the event policy in thisfield.

l Enable Event Policy. This toggle allows you to enable and disable the event policy.

l Policy Description. In this field, type a description of the event policy.

Creating an Event Policy for Agent Logs

Page 70: Monitoring with the SL1 Agent (Version 10.2.0)

Creating an Event Policy for Agent Logs

4. Click the [Match Logic] tab to define pattern-matching for the event:

5. Complete the following fields:

l Event Source. Select ScienceLogic Agent as the source for the event. The fields below this field willchange based on your selection.

l Log Policy. Select the Log File Monitoring Policy the agent will use to collect the log message.

6. After selecting and defining your Event Source, enter values in the fields on the right side of theMatchLogic tab specify the text that must appear in the log message for the event policy to trigger:

l String/Regular Expression. Use this dropdown to select String or Regular Expression.

l Match String. A string used to correlate the event with a log message. Can be up to 512 charactersin length. To match this event policy, the text of a log message or alert must match the value youenter in this field. Can be any combination of alpha-numeric and multi-byte characters. SL1'sexpression matching is case-sensitive. This field is required for events generated with a source ofSyslog, API, and Email.

l Second Match String (Optional). A secondary string used to match against the originating logmessage. Can be up to 512 characters in length. Can be any combination of alpha-numeric andmulti-byte characters. To match this event policy, the text of a log message or alert must match thevalue you enter in this field and the value you entered in theMatch String field. This field is optional.

l Allow event to expire if it doesn't reoccur within a time frame. If toggled on, enter the time inwhich an active event will be cleared automatically if there is no reoccurrence of the event in the fieldsthat appear. You can enter time in minutes or hours.

l Require multiple triggers within a time frame. If toggled on, enter the number of events and thetime in which an event requires multiple triggers to occur in the fields that appear. You can enter timein minutes or hours.

70

Page 71: Monitoring with the SL1 Agent (Version 10.2.0)

71

l Detection Weight. If two event definitions are very similar, the weight field specifies the order inwhich SL1 should match messages against the similar event definitions. The event definition with thelowest weight will be matched first. This field is most useful for events that use expression matching.Options range from 0 (first) - 20 (last).

l Multi Match. By default, SL1 will match a log message or alert to only one event policy. If a logmessage or alert matches multiple event polices, SL1 will use the Detection Weight setting todetermine which event policy the log message or alert will match. If you select theMulti Matchcheckbox in all event policies that can match the same log message or alert, SL1 will generate anevent for every event policy that matches that single log message or alert.

l Message Match. If SL1 has generated an event and then a second log message or alert matches thesame event policy for the same entity, SL1 will not generate a second event, but will increase thecount value for the original event on the Events page and in the Viewing Events page. By default,this behavior occurs regardless of whether the two log messages or alerts contain the same message.If you select theMessage Match checkbox, this behavior will occur only if the log messages or alertscontain the same message.

7. Click the [Event Message] tab to configure the event:

8. Complete the following fields:

Creating an Event Policy for Agent Logs

Page 72: Monitoring with the SL1 Agent (Version 10.2.0)

Creating an Event Policy for Agent Logs

l Event Message. The message that appears in the Event Console page or the Viewing Eventspage when this event occurs. This field defaults to "%M" for new event policies upon creation. Themessage can be any combination of alphanumeric and multi-byte characters. Variables include thecharacters "%" (percent) and "|" (bar). You can also use regular expressions and variables thatrepresent text from the original log message to create the Event Message.To include regular expressions in the Event Message, surround the regular expression with%R and%/R. For example,%RFilename: .*? %/R would search for the first instance of the string "Filename:" (Filename-colon-space) followed by any number of any characters up to the line break. The %Rindicates the beginning of a regular expression. The %/R indicates the end of a regular expression.SL1 uses the regular expression to search the log message and use the matching text in the eventmessage. For details on the regular expression syntax allowed by SL1, seehttp://www.python.org/doc/howto/.

You can also use the following variables in the Event Message field:

o %I ("eye"). This variable contains the value that matches the Identifier Pattern field in the[Advanced] tab.

o %M. The full text of the log message that triggered the event will be displayed in EventMessage field.

o %T. Threshold value from the log file will be displayed in Event Message field.

l Event Severity. Defines the severity of the event. Choices are:

o Healthy. Healthy events indicate that a device or condition has returned to a healthy state.Frequently, a healthy event is generated after a problem has been fixed.

o Notice. Notice events indicate a condition that does not affect service but about which usersshould be aware.

o Minor. Minor events indicate a condition that does not currently impair service, but thecondition needs to be corrected before it becomes more severe.

o Major. Major events indicate a condition that impacts service and requires immediateinvestigation.

o Critical. Critical events indicate a condition that can seriously impair or curtail service andrequires immediate attention (i.e., service or system outages).

l Use Modifier. If selected, when the event is triggered, SL1 will check to see if the interface associatedwith this event has a custom severity modifier. If so, the event will appear in the Event Console withthat custom severity modifier applied to the severity in the Event Severity field. For example, if aninterface with an Event Severity Adjust setting of Sev -1 triggers an event with an Event Severity ofMajor and that event has the Use Modifier checkbox selected, the event will appear in the EventConsole with a severity ofMinor.

9. Optionally, supply values in the other fields on this page. For more information on the remaining fields, aswell as the [Suppressions] tab, see the Defining and Editing Event Policies chapter in the Eventsmanual.

10. Click [Save].

72

Page 73: Monitoring with the SL1 Agent (Version 10.2.0)

73

Creating an Event Policy for Agent Logs in the Classic UserInterface

To trigger events based on log messages collected by the agent in the classic user interface for SL1, you mustcreate an event policy that is associated with a Log File Monitoring policy.

To create an event policy in the classic user interface based on log data collected by the agent:

1. Go to Event Policy Manager page (Registry > Events > Event Manager in the classic user interface).

2. In the Event Policy Manager page, click [Create]. The Event Policy Editor page appears:

3. In the Event Policy Editor page and set of tabs, you can define a new event. The Event Policy Editor pagecontains three tabs:

l Policy. Define basic parameters for the event.

l Advanced. Define pattern-matching for the event and also define event roll-ups and suppressions.

l Suppressions. Suppress the event on selected devices. When you suppress an event, you arespecifying that, in the future, if this event occurs again on a specific device, the event will not appearin the Event Console page or the Viewing Events page for the device.

4. Supply values in the following fields:

l Event Source. Select ScienceLogic Agent.

Creating an Event Policy for Agent Logs in the Classic User Interface

Page 74: Monitoring with the SL1 Agent (Version 10.2.0)

Creating an Event Policy for Agent Logs in the Classic User Interface

l Policy Name. The name of the event. Can be any combination of alphanumeric characters, up to48 characters in length.

l Operational State. Specifies whether event is to be operational or not. Choices are Enabled orDisabled.

l Event Message. The message that appears in the Event Console page or the Viewing Eventspage when this event occurs. Can be any combination of alphanumeric characters.

o You can use regular expressions that represent text from the original log message to create theEvent Message:

n %R. Indicates a regular expression. Surround the regular expression with %R and %/R. Forexample, %RFilename: .*? %/R would search for the first instance of the string "Filename: "followed by any number of any characters up to the line break. For details on the regularexpression syntax allowed by SL1, see http://www.python.org/doc/howto/.

n %I ("eye"). This variable contains the value that matches the Identifier Pattern field in the[Advanced] tab.

n %M. The full text of the log message that triggered the event will be displayed in EventMessage field.

n %T. Threshold value from the log file will be displayed in Event Message field.

l Event Severity. Defines the severity of the event. Choices are:

o Healthy. Healthy Events indicate that a device or condition has returned to a healthy state.Frequently, a healthy event is generated after a problem has been fixed.

o Notice. Notice Events indicate a condition that does not affect service but about which usersshould be aware.

o Minor. Minor Events indicate a condition that does not currently impair service, but the conditionneeds to be corrected before it becomes more severe.

o Major. Major Events indicate a condition that is service impacting and requires immediateinvestigation.

o Critical. Critical Events indicate a condition that can seriously impair or curtail service and requireimmediate attention (i.e. service or system outages).

l Use Modifier. If selected, when the event is triggered, SL1 will check to see if the interface associatedwith this event has a custom severity modifier. If so, the event will appear in the Event Console withthat custom severity modifier applied to the severity in the Event Severity field. For example, if aninterface with an Event Severity Adjust setting of Sev -1 triggers an event with an Event Severity ofMajor and that event has the Use Modifier checkbox selected, the event will appear in the EventConsole with a severity ofMinor.

l Policy Description. Text that explains what the event means and what possible causes are.

5. Select the [Advanced] tab.

6. In the Log Policy field, select the Log File Monitoring policy that the agent will use to collect the logmessage.

74

Page 75: Monitoring with the SL1 Agent (Version 10.2.0)

75

7. Enter values in the following fields to specify specific text that must appear in the log message for the eventpolicy to trigger:

l First Match String. A string used to match against the originating log message. To match this eventpolicy, the text of a log message must match the value you enter in this field. Can be any combinationof alphanumeric characters. Expression matching in SL1 is case-sensitive.

l Second Match String. A secondary string used to match against the originating log message. Tomatch this event policy, the text of a log message must match the value you enter in this field and thevalue you entered in the First Match String field. This field is optional.

NOTE: TheMatch Logic field specifies whether SL1 should process First Match String and Second MatchString as simple text matches or as regular expressions.

8. Optionally, supply values in the other fields on this page. For more information on the remaining fields, aswell as the [Suppressions] tab, see the Eventsmanual.

9. Click [Save].

Creating an Event Policy for Agent Logs in the Classic User Interface

Page 76: Monitoring with the SL1 Agent (Version 10.2.0)

Chapter

5Agent Troubleshooting

Overview

This chapter contains troubleshooting processes that you can use to address issues with the SL1 agent.

TIP: On a Windows agent, you can run the following diagnostic command to generate a *.tar.gz file thatcontains useful information for troubleshooting issues: ...\SiloAgent.exe --diag

TIP: On a Linux agent, use –-diag for the diagnostic option.

Use the following menu options to navigate the SL1 user interface:

l To view a pop-out list of menu options, click the menu icon ( ).

l To view a page containing all the menu options, click the Advanced menu icon ( ).

To troubleshoot potential issues with SL1 agents, perform the following procedures, in the following order:

Was the Agent Download Successful? 78

Is the Agent Process Running? 78

Is the Agent Configuration Valid? 79

Has SL1 Discovery Completed? 79

Is the Agent Able to Upload Data? 80

Is SL1 Receiving Agent Data? 81

Can SL1 Process Agent Data? 82

76

Page 77: Monitoring with the SL1 Agent (Version 10.2.0)

77

Is the Number of Processes Inconsistent with Other Applications? 82

Does MySQL on the Collector Unit Have a Connection Issue? 83

Troubleshooting Examples 83

Additional Troubleshooting Situations and Best Practices 84

Agent Communication with SL1 85

Page 78: Monitoring with the SL1 Agent (Version 10.2.0)

Was the Agent Download Successful?

Was the Agent Download Successful?

If the Windows agent download failed with a "500 Internal Server Error", restart the uwsgi service on the SL1system from which you are downloading the agent.From an administrator command prompt in Windows, run the following command on the SL1 system:

systemct restart uwsgi

Is the Agent Process Running?

As a first step, always locate the following logs from the Message Collectors when troubleshooting:

l /var/log/streamer_prime/streamer_prime.log

l /var/log/uwsgi/streamer_prime.log

To determine if the agent process is running:

1. Check the Windows Task Manager or run the "tasklist" or "top" command, and look for SiloAgent.exe(Windows) or scilogd (Linux).

l Windows: If SiloAgent.exe is not running, check the "Application" event log for events withsource=SiloAgent.

l Linux: If scilogd is not running, check /var/log/messages or /var/log/syslog for relevant logmessages.

If you are using the SL1 Extended Architecture, determine if the agent was deleted from the [Agents] tab insteadof uninstalling the agent.

If the agent was deleted in the SL1 user interface, SL1 shuts down the agent instead of uninstalling theagent. You should re-install the agent that you deleted in SL1.

To re-install the agent that was shut down:

1. Uninstall the agent that you shut down.

2. Delete that agent's configuration from one of the following locations:

l Windows: C:\Program Files\ScienceLogic\SiloAgent\conf\scilog.conf

l Linux: /etc/scilogd/scilog.conf

3. Install a new agent.

If the agent was not deleted, then the issue could be with the agent. You should generate diagnosticsinformation to share with your ScienceLogic contact.

To generate diagnostics information for an agent:

1. From an administrator command prompt, run one of the following commands:

78

Page 79: Monitoring with the SL1 Agent (Version 10.2.0)

79

l Windows: C:\Program Files\ScienceLogic\SiloAgent\bin\SiloAgent.exe -diag

l Linux: /usr/bin/scilogd --diag

2. Share the contents of the newly created diagnostic file in the current directory with your ScienceLogiccontact. Depending on your operating system, the file name is:

l Windows: scilog-<current date>.diag.tgz

l Linux: sl-diag.tar.gz

Is the Agent Configuration Valid?

1. Check the agent configuration in one of the following locations:

l Windows: C:\Program Files\ScienceLogic\SiloAgent\conf\scilog.conf

l Linux: /etc/scilogd/scilog.conf

2. Check the configuration item CollectorID:

l If there is no CollectorID tag, then the agent has not been able to reach the stream or messagecollector.

l The CollectorID should be a GUID similar to "4179b06ef502129c3023a0f8d58f3c37".

3. Check the configuration item URLfront, which is where the agent attempts to get the configuration file.

l Determine if you can ping the URLfront.

l If you are using a distributed architecture, URLfront should be the URL of the Message Collector.

l If you are using the SL1 Extended Architecture, URLfront should be the URL of the Streamercontainer, such as pod9-streamer0.

l If the URL for URLfront is not correct, then re-install the agent.

l If the URL for URLfront is correct, then determine if you can ping the host portion of URLfront. If youcannot ping, then there are customer firewall or NAT issues.

Has SL1 Discovery Completed?

To check if SL1 discovery finished:

1. Check the agent record in the distributed architecture by using SSH to communicate with the MessageCollector and running the following commands:

redis-cli -p 6380keys *hgetall agent_<GUID>

where<GUID> is the specific value from scilog.conf.

Is the Agent Configuration Valid?

Page 80: Monitoring with the SL1 Agent (Version 10.2.0)

Is the Agent Able to Upload Data?

2. Look for a field "did". The following line is the EM7-device-id.

3. If the EM7-device-id is not present, then discovery has not completed.

4. Alternately, you can look on the Devices page for the device to compare the shown device-id to the valuein the agent record.

Is the Agent Able to Upload Data?

Check the Agent Upload Directory

Check the upload directory for the agent for directories and files in one of the following locations:

l Windows: C:\Program Files\ScienceLogic\SiloAgent\data

l Linux: opt/scilog/data

If there are many items, then the agent is unable to upload.

If the number of items is decreasing, the agent might have an issue. The agent is slowly catching up, but thissituation indicates that a previous issue existed.

If the number of items continues to increase overall, check the configuration item URL:

l The URL is the location where the agent attempts to upload files.

l Determine if the host portion of the URL is reachable. If the host portion is reachable, the name of the oldestitem indicates the approximate time of the issue.

NOTE: To prevent consuming the disk with backed-up data, the agent limits the size and count of items in theupload directory.

A procedural note regarding backed-up data:

For a new installation, the agent reaches out to the streamer for a configuration file. If the configuration file can’treach the streamer, the streamer goes into a slow poll mode, waiting for a good configuration file. In themeantime, the streamer does nothing else (it does not generate data or log files). As a result, even through itlooks like there is no backup of data files, in reality, there are no data files.

After the Streamer container receives a valid configuration file:

l After a restart, the agent reaches out to the Streamer container for a new configuration file.

l If the agent can't reach the Streamer, the agent will still generate data files, because it has a validconfiguration file from a previous run. In this situation, you will see data files backing up if the Streamer isunreachable.

In summary, if you have a valid configuration, you will get data files. If you do not have a backup, Streamer canbe reached.

80

Page 81: Monitoring with the SL1 Agent (Version 10.2.0)

81

Run the Agent in Debug Mode (Linux)

NOTE: You might need to preface the following commands with sudo if you are in root-privileged mode.

1. Stop the agent daemon by running the following command:

service scilogd stop

2. Start the agent from the command line:

scilogd -d 2>&1 | tee /tmp/scilogd.log

3. Let the agent run for about five minutes.

4. Press Ctrl+C and examine the output file.

5. Restart the agent by running the following command:

service scilogd start

Is SL1 Receiving Agent Data?

If you are using a distributed architecture:

1. SSH into the Message Collector and run the following command:

sudo tail -n 100 /var/log/uwsgi/streamer_prime_uwsgi.log

2. Look for lines containing the hostname of the monitored device, such as the following:

10.2.16.40 - - [19/Apr/2018:17:04:55 +0000] "POST /SaveData.py/save_dataHTTP/1.1" 200 59 "-" "Windows SiloAgent : aym-win2012r2-0"

3. If there are no matching lines, then the Streamer container is not getting data from that agent.

If you are using the SL1 Extended Architecture:

1. SSH into the Management Node and view the logs.

2. Look for lines containing the HOSTNAME of the monitored device.

3. If there are no matching lines, then the Streamer container is not getting data from that agent.

Is SL1 Receiving Agent Data?

Page 82: Monitoring with the SL1 Agent (Version 10.2.0)

Can SL1 Process Agent Data?

Can SL1 Process Agent Data?

Check the Message Collector log files or SL1 Streamer log files:

1. If you are using the distributed architecture, locate the following files from the SL1 message collector andprovide the files to your ScienceLogic contact:

l /var/log/uwsgi/streamer_prime_uwsgi.log

l /var/log/streamer_prime/streamer_prime.log

2. If you are using the SL1 Extended Architecture, check all logs for ERR or Except:

for p in $(kubectl get pods | cut -f 1 -d ' '); do echo $p; kubectl logs $p |grep -E "(ERR|Except)"; done

3. Search for "ERROR", "Exception", and "HARAKIRI".

4. Contact your ScienceLogic contact with any error messages you find in the log files.

NOTE: If you do not find any error messages, then the issue is most likely with the DynamicApplication that runs on the data collector.

Is the Number of Processes Inconsistent with OtherApplications?

l On Linux, many outputs from the ps command list the kernel threads (the processes listed in squarebrackets). Because the agent is not in the kernel, it will not list kernel threads.

l Be aware that the agent reports processes that are running as well as processes that started and may havestopped, while top or ps commands show processes that exist when they are executed.

l Check the agent configuration. Due to back-end space limitations, many configuration combinations canlimit what data the agent sends. A combination of parameters to get all processes include the following:

l NIPD True. The agent library can not get into all processes at times, often on install. Non-intercepted process discovery reports processes that are not intercepted via the library.

l SLPAggregation. This parameter takes short-lived processes that exist for less than 80 seconds androlls information about the processes into the information for their parents. As a result, the short-livedprocesses will not be seen.

82

Page 83: Monitoring with the SL1 Agent (Version 10.2.0)

83

Does MySQL on the Collector Unit Have a Connection Issue?

If MySQL on the Collector Unit has more connections than the configured limit, the Collector Unit will raise aconnection issue and cause the agent connection failure. The following error message is an example of thissituation:

Unhandled exception: Traceback (most recent call last):File "/opt/em7/backend/process/proc_pda.py", line 223, in __init__File "/opt/em7/backend/silo_collect/streamer/main.py", line 353, in mainFile "/opt/em7/backend/silo_collect/shared.py", line 108, in run_collectionFile "/opt/em7/backend/silo_collect/shared.py", line 156, in _run_collect_asyncFile "/usr/lib64/python2.7/multiprocessing/pool.py", line 554, in get raise self._value ChildProcessException: Traceback (most recent call last):File "/opt/em7/backend/silo_common/misc.py", line 520, in new_f File"/opt/em7/backend/silo_collect/streamer/main.py", line 225, in streamer_collectFile "/opt/em7/backend/silo_common/database.py", line 701, in local_dbFile "/opt/em7/backend/silo_common/database.py", line 92, in callFuncFile "/opt/em7/backend/silo_common/database.py", line 693, in get_dbc MySQLError:Error attempting to connect to database with SSL enabled False: (2013, 'Lostconnection to MySQL server at \'reading initial communication packet\', systemerror: 0 "Internal error/check (Not system error)"')

To address this connection issue:

1. To determine the current connection limit, run the following command on the Collector Unit:

silo_mysql -e 'show variables like "max_connections";'

2. To view the high watermark for concurrently active connections, run the following command on theCollector Unit:

show status where 'Variable_name' = 'max_used_connections';

3. If the value is more than the limit value, try to dynamically but temporarily increase the limit. For example:

silo_mysql -e 'set global max_connections=350'

Troubleshooting Examples

Example /var/log/streamer_prime/streamer_prime.log for successful discovery

2019-01-04T17:07:42.355291+00:00 amateen-em7 journal: SCILO_SP:6954|logger:log_info:132|INFO|Agent config request received with init flag set to True. GeneratedTemp AID: 2ae22a6b4489457abb14373cd3816076. Request: <WSGIRequest: GET'/api/collector/config/?collector_key=aEf34$aq3TGSDdf&tenant_id=0&host_name=aym-win2012r2-1&init=&os=windows&collector_id=0'>

2019-01-04T17:07:42.619082+00:00 amateen-em7 journal: SCILO_SP:6954|logger:log_info:132|INFO|Calling Agent version with: <QueryDict: {'collector_id':['2ae22a6b4489457abb14373cd3816076'], 'type': ['windows_64'], 'tenant_id': ['0'],'host_name': ['aym-win2012r2-1'], 'collector_key': ['aEf34$aq3TGSDdf'], 'version':['115']}>

Does MySQL on the Collector Unit Have a Connection Issue?

Page 84: Monitoring with the SL1 Agent (Version 10.2.0)

Additional Troubleshooting Situations and Best Practices

2019-01-04T17:07:43.028457+00:00 amateen-em7 journal: SCILO_SP:16717|logger:log_warning:127|WARNING|System file received from aym-win2012r2-1

2019-01-04T17:07:43.032897+00:00 amateen-em7 journal: SCILO_SP:16717|logger:log_info:132|INFO|Making discovery call for agent 2ae22a6b4489457abb14373cd3816076

2019-01-04T17:07:43.746284+00:00 amateen-em7 journal: SCILO_SP:30843|logger:log_warning:127|WARNING|System file received from aym-win2012r2-1

2019-01-04T17:07:43.750553+00:00 amateen-em7 journal: SCILO_SP:30843|logger:log_warning:127|WARNING|Discovery call within time threshold, sleeping.

2019-01-04T17:07:46.676827+00:00 amateen-em7 journal: SCILO_SP:16717|logger:log_info:132|INFO|Update agent request did: 4, oid: 0, ip: 10.7.6.119, agent id:2ae22a6b4489457abb14373cd3816076

2019-01-04T17:07:46.677114+00:00 amateen-em7 journal: SCILO_SP:16717|logger:log_info:132|INFO|Discovery complete, getting new agent device id. Downloading newconfig for device: 2ae22a6b4489457abb14373cd3816076.

2019-01-04T17:07:47.420509+00:00 amateen-em7 journal: SCILO_SP:6954|logger:log_warning:127|WARNING|Agent id: 2ae22a6b4489457abb14373cd3816076 being given areturn code: 2

Example /var/log/uwsgi/streamer.log for successful discovery in the distr ibutedarchitecture

10.234.196.19 - - [29/Sep/2017:14:04:52 +0000] "POST /api/update_agent/agent/HTTP/1.1" 200 59 "-" "python-requests/2.7.0 CPython/2.7.5 Linux/3.10.0-514.10.2.el7.x86_64"

Save incoming data for a specif ic device ID in the distr ibuted architecture

PYTHONPATH=/opt/em7/lib/python3:/opt/streamer_prime python3 /opt/streamer_prime/streamer_prime/manage.py agent_save_xml -d <agent guid> -e true

Save incoming data for a specif ic device ID in the SL1 Extended Architecture

kubectl exec -it $(kubectl get pods -l app=streamer -o jsonpath="{.items[0].metadata.name}") -- python -m streamer agent_save_data --host_id <host id> --enable true

You can find the host id from the ADS url, such as https://<sl1_address>/ads/servers/13/system). You canlocate the files in the /tmp/save_agent_data directory.

Additional Troubleshooting Situations and Best Practices

The following situations might occur while configuring or working with agents:

84

Page 85: Monitoring with the SL1 Agent (Version 10.2.0)

85

Situation Cause / Resolution

Two device records exist for the same device on theDevices page in SL1.

This situation occurs when SL1 first discovered thisdevice with SNMP, and then the agent was installedand started polling that device. This duplication ofrecords also occurs if the agent was installed first, andthen you ran an SNMP discovery.

To address this issue, you canmerge the devicerecords using the classic user interface. For moreinformation, see the "Managing a Single Device withthe Device Administration Panel" chapter in theDevice Managementmanual.

The SNMP device record has IPv4, but the agentdevice record has IPv6.

The agent reports all network interfaces to themessage collector. The Message Collector uses thefirst "bound" IP address reported by the agent.

To address this issue, you can manually edit the agentdevice record in the "classic" user interface andupdate the IP address.

If you uninstall an agent and then run a differentinstallation executable file, you still see the sameorganization ID for the agent record.

After you uninstall the agent, the scilog.conf file is lefton the server in case the agent is reinstalled. SL1 canreuse the same device record and maintain historicalperformance data for that agent.

To address this issue, delete the scilog.conf file afteryou run the uninstallation. If you install this agentagain, SL1 assigns a new organization ID to the agentand creates a new device record.

Agent Communication with SL1

This section covers the various codes that are sent to the SL1 agent when additional actions are needed.

Return Codes

The core method of communication between the SL1 agent and SL1 involves the data files the agent uploads tothe Streamer. When the agent uploads a file to the Streamer, a number of conditions are checked and the returncode given to the agent determines what action should be taken, if any.

Return Code Meaning

200 No action needed.

409 The data file uploaded by the agent has a problem. The Streamer did not accept it and itshould be deleted from the host

Agent Communication with SL1

Page 86: Monitoring with the SL1 Agent (Version 10.2.0)

Agent Communication with SL1

Return Code Meaning

503 Data uploads are occurring more quickly than normal. The agent should refrain fromsending its next file until a specified amount of time has passed.

505 The agent should stop streaming.

Sub-codes

The Streamer communicates another set of actions in a JSON response included with the return code to the SL1agent.

JSON Response Field Type Meaning

send_system_file boolean Send a system config file as the next upload from this agent.

get_agent_config boolean Retrieve an updated agent config from the Streamer.

get_agent_update boolean Download a new version of the agent to update.

get_pd_config boolean Retrieve an updated polled data config.

get_log_config boolean Retrieve an updated log config.

purge_uploads boolean Delete any backed-up data and only upload new files.

set_data_backoff integer Do not upload a data file until the set number of seconds havepassed.

set_log_backoff integer Do not upload a log file until the set number of seconds havepassed.

86

Page 87: Monitoring with the SL1 Agent (Version 10.2.0)

© 2003 - 2021, ScienceLogic, Inc.

All rights reserved.

LIMITATION OF LIABILITY AND GENERAL DISCLAIMER

ALL INFORMATION AVAILABLE IN THIS GUIDE IS PROVIDED "AS IS," WITHOUT WARRANTY OF ANYKIND, EITHER EXPRESS OR IMPLIED. SCIENCELOGIC™ AND ITS SUPPLIERS DISCLAIM ALL WARRANTIES,EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OFMERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT.

Although ScienceLogic™ has attempted to provide accurate information on this Site, information on this Sitemay contain inadvertent technical inaccuracies or typographical errors, and ScienceLogic™ assumes noresponsibility for the accuracy of the information. Information may be changed or updated without notice.ScienceLogic™ may also make improvements and / or changes in the products or services described in thisSite at any time without notice.

Copyrights and Trademarks

ScienceLogic, the ScienceLogic logo, and EM7 are trademarks of ScienceLogic, Inc. in the United States,other countries, or both.

Below is a list of trademarks and service marks that should be credited to ScienceLogic, Inc. The ® and ™symbols reflect the trademark registration status in the U.S. Patent and Trademark Office and may not beappropriate for materials to be distributed outside the United States.

l ScienceLogic™l EM7™ and em7™l Simplify IT™l Dynamic Application™l Relational Infrastructure Management™

The absence of a product or service name, slogan or logo from this list does not constitute a waiver ofScienceLogic’s trademark or other intellectual property rights concerning that name, slogan, or logo.

Please note that laws concerning use of trademarks or product names vary by country. Always consult alocal attorney for additional guidance.

Other

If any provision of this agreement shall be unlawful, void, or for any reason unenforceable, then thatprovision shall be deemed severable from this agreement and shall not affect the validity and enforceabilityof any remaining provisions. This is the entire agreement between the parties relating to the matterscontained herein.

In the U.S. and other jurisdictions, trademark owners have a duty to police the use of their marks. Therefore, if you become aware of any improper use of ScienceLogic Trademarks, including infringement orcounterfeiting by third parties, report them to Science Logic’s legal department immediately. Report asmuch detail as possible about the misuse, including the name of the party, contact information, and copiesor photographs of the potential misuse to: [email protected]

Page 88: Monitoring with the SL1 Agent (Version 10.2.0)

800-SCI-LOGIC (1-800-724-5644)

International: +1-703-354-1010