are you a smart connector? - hewlett packard enterprise · are you a smart connector? mark ulmer,...

Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Are you a smart connector? Mark Ulmer, CISSP, SCJP SIEM Engineer TIAA-CREFF

2

About TIAA-CREF

We are TIAA-CREF

Founded in 1918. Were dedicated to serving the lifelong financial needs of those in the academic, medical, cultural, governmental and research fields.

Fact Facts

Audience Who is this talk for?

ArcSight analysts and administrators Those who need to keep ArcSight well fed Focus on SmartConnector software for Windows & Unix/Linux

SmartConnector version We assume version 5.2.2 and higher (prompts for certificate)

Participation

I dont mind questions, yet please be mindful of time Theres always the hallways and email

Agenda

Basics Connector installs Have a Connector Appliance? Anatomy of a connector folder Running as a service Config files

agent.properties file agent.default.properties file

Log files Reading the agent.log Reading the

agent.out.wrapper.log

Tuning Logging levels Connector JVM memory

Analysis Running Test Alerts Running RegEx Running LogFu on connector logs Running Flex Agent Wizard Tips Command line Get Status Knowledge documents Silent Install headaches Connector service removal Clean up time Q & A

DISCLAIMER: The configurations and information described herein is provided on an "as is" basis, without warranty of any kind, to the fullest extent permitted by law. Mark Ulmer, TIAA-CREF and HP do not warrant or guarantee the individual success of any persons that may have in implementing the sample configurations on their own platforms or using in their own environments. Mark Ulmer, TIAA-CREF and HP do not warrant, guarantee or make any representations regarding the use, results of use, accuracy, timeliness or completeness of any data or information relating to the configurations. Mark Ulmer, TIAA-CREF and HP disclaims all warranties, express or implied, and in particular, disclaims all warranties of merchantability, fitness for a particular purpose, and warranties related to the configurations, or any service or software related thereto. Mark Ulmer, TIAA-CREF and HP shall not be liable for any direct, indirect or consequential damages or costs of any type arising out of any action taken by you or others related to the configurations and information.

Demo SmartConnector install on Windows

Ref: http://www.youtube.com/watch?v=yqlymqqICuM

http://www.youtube.com/watch?v=yqlymqqICuM

Demo SmartConnector install on Linux (command line)

Demo SmartConnector upgrades on Connector Appliance

Tip: Have a SmartConnector Appliance?

Major benefit of having at least one in your environment: Centralized Management of most SmartConnectors

Enabling Remote Management on SmartConnector Ref: http://support.openview.hp.com/selfsolve/document/KM1272694

http://support.openview.hp.com/selfsolve/document/KM1272694

SmartConnector running as a service

Windows Linux Linux command line: service arc_bluecoat_file stop |start | restart or /etc/init.d/arc_bluecoat_file stop |start | restart

Linux Tip: the locate command

From Wikipedia, the free encyclopedia

locate is a Unix utility first created in 1983 used to find files on filesystems. It searches through a prebuilt database of files generated by updatedb or a daemon and compressed using incremental encoding. It is significantly faster than find, but requires the database to be updated regularly.

Agenda







SmartConnector agent.properties

Become familiar with your \user\agent\agent.proerties file. These are your configuration choices made during installation. Backup this file.

#ArcSight Properties File #Fri Apr 19 16:03:08 EDT 2013 agents.maxAgents=1 agents[0].AgentSequenceNumber=0 agents[0].confighome=\\..\\bluecoat_file agents[0].destination.count=1 agents[0].destination[0].agentid=3ArAKHGFY4AQ0XHJKHOg\=\= agents[0].destination[0].params=\n\n

agent.properties (continued) Support Knowledge document: KM1271052

agents[0].logfilehome=d\:\\FtpBlueCoatLogs agents[0].onrotation=DeleteFile agents[0].onrotationoptions=processed agents[0].persistenceinterval=0 agents[0].preservedstatecount=10 agents[0].preservedstateinterval=30000 agents[0].preservestate=false agents[0].rotationdelay=30 agents[0].skipabnormalfile=false agents[0].startatend=false agents[0].type=bluecoat_file # Added by me on 6/9/2013 to reduce logging messages. # 0 = Debug, 1 = Info, 2 = Warn, 3 = Error, 4 = Fatal log.channel.file.property.package.com.arcsight=2

Onrotation options ** None RenameFileInTheSameDirectory DeleteFile

Hey what is this?

Ref: http://support.openview.hp.com/selfsolve/document/KM1271052


SmartConnector agent.default.properties

To review and understand the connector settings and defaults you can VIEW the \config\agent\agent.default.properties file.

DO NOT EDIT THIS FILE. Copy a property line you want modify over to your agent.properties file. E.g. Number of agent.log backup files is 10 by default. Lets make it 20.

#ArcSight Properties File #Tue Jun 07 17:41:02 MST 2011 . . . # Added by me on --DATE-- to have more logging history. # The maximum number of agent.log backup files

log.channel.file.property.maxbackupindex=20

# ============================================================ # ArcSight Smart Agent default properties # ============================================================ # ============================================================ # Log configuration. # ============================================================ # The loglevel for the default package. Anything with a level # >= the one specified will be logged. # 0 = Debug, 1 = Info, 2 = Warn, 3 = Error, 4 = Fatal log.channel.file.property.package.com.arcsight=1 # The path and name of the log file. log.channel.file.property.path=agent.log # The maximum size of the log file before it will be rolled over. log.channel.file.property.maxsize=10MB # The maximum number of backup files to create for rolling over.

log.channel.file.property.maxbackupindex=10

Agenda







Reading the agent.log file

[2013-08-02 17:44:56,606][INFO ][run] Memory Usage: 649Mb out of 1004Mb [2013-08-02 17:44:56,606][INFO ][logStatus] {Agent Type=bluecoat_file, Agent Version=6.0.2.6627.0, Event rate

LTC=Fri Aug 02 17:43:56 EDT 2013, Events Processed=238421888, Events Processed(SLC)=36711, Events/Sec=477.7897998036112, Events/Sec(SLC)=622.2203389830509, FCP Version=0, FIPS Enabled=false, First Event Processed=Sat Jul 27 23:07:17 EDT 2013, Host Address=10.1.1.54, Host Name=SmartConnector-Server1.company.com, Last Event Processed=Fri Aug 02 17:44:08 EDT 2013, activeThreadCount=77, logfilehome=e:\FtpBlueCoatLogs, logparamlist.count=2, logparamlist[0].initialfilename=SG_arcsight__8976432356.log.gz, logparamlist[0].logfilenameformat=SG_arcsight__89.+gz, logparamlist[0].logfiletype=main, logparamlist[1].initialfilename=SG_arcsight__3278614456.log.gz, logparamlist[1].logfilenameformat=SG_arcsight__32.+gz, logparamlist[1].logfiletype=main}

[2013-08-02 17:44:56,606][INFO ][logStatus] {Eps=622.2203389830509} [2013-08-02 17:44:56,606][INFO ][logStatus] {C=0, ET=Up, HT=Up, N=Webproxy, S=136763051, T=723.8813} [2013-08-02 17:44:56,606][INFO ][logStatus] {Last Start Time=1374980835937, Uptime=499060} [2013-08-02 17:44:58,445][INFO ][checkAndFollowRotatedFile] The previous file is

[e:\FtpBlueCoatLogs\SG_arcsight__80756802213953.log.gz]. Available file name is [e:\FtpBlueCoatLogs\SG_arcsight__80876340802214453.log.gz]. Last updated [Fri Aug 02 17:44:57 EDT 2013]. Rotated [true]. The current length is [22351]

[2013-08-02 17:44:58,445][INFO ][checkAndFollowRotatedFile] The file [SG_arcsight__80.+gz] has been rotated. Starting afresh. New file name is [d:\FtpBlueCoatLogs\SG_arcsight__180876340802214453.log.gz]. Last modified [Fri Aug 02 17:44:57 EDT 2013] The current length is 22351, the previous length is 0

[2013-08-02 17:44:58,445][INFO ][][fileEnded] Finished processing file [SG_arcsight__ 80756802213953.log.gz]. Status: success

Reading the agent.log - Log Status Support Knowledge document: KM1262122

C = If this value is non-zero, then the SmartConnector is caching. S = Running total of events which have been sent to the Manager since the SmartConnector was last started. T = Shows the average number of events being passed to the Manager per second (Throughput). HT = Heartbeat Transport. This gives the status of the connection between SmartConnector and Manager. ET = Event Transport. This indicates whether the Manager is accepting events from the SmartConnector. If this is Down, it means that the Manager has paused the SmartConnector. N = Name of the SmartConnector Sample logStatus from agent.log: 2010-05-15 12:29:20,347][INFO ][default.com.arcsight.agent.hi][logStatus] {C=308, ET=Up, HT=Up, N= Syslog, S=661530, T=2.566666666666667} http://support.openview.hp.com/selfsolve/document/KM1262122


agent.log (file continued)

[2011-07-24 14:54:18,965][INFO ][default.com.arcsight.agent.loadable._EventCounter][processSingleAlert]

First event from [Unix|Unix||10.12.10.127] received. [2011-07-24 14:54:18,977][INFO

][default.com.arcsight.agent.loadable._DeviceEventCounter][processSingleAlert] New device found [cc34cc0ac8.company.com|10.12.18.27|Unix|Unix]. Starting counters. [2011-07-24 15:19:33,782][WARN ][default.com.arcsight.agent.wf.i][run] Unable to find subagent for with

message :.. Warnings logged [15560] [2011-07-24 15:19:33,782][WARN ][default.com.arcsight.agent.wf.i][run] Subagent not found for message

[]. Warnings logged [15561]

[2011-07-24 15:19:33,782][ERROR][default.com.arcsight.agent.wf.k][processMsg] java.lang.NumberFormatException: For input string: "/tr" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48) at java.lang.Integer.parseInt(Integer.java:449) at java.lang.Integer.parseInt(Integer.java:499)

Reading the agent.out.wrapper.log file

INFO | jvm 1 | 2013/08/02 16:04:19 | [Fri Aug 02 16:04:19 EDT 2013] [INFO ] {Eps=0.01694915254237288, Evts=234396739}

INFO | jvm 1 | 2013/08/02 16:04:19 | [Fri Aug 02 16:04:19 EDT 2013] [INFO ] {C=0, ET=Up, HT=Up, N=Webproxy1, S=134123034, T=0.23728813559322035}

INFO | jvm 1 | 2013/08/02 16:04:20 | [GC 998849K->902664K(1012672K), 0.0101251 secs] INFO | jvm 1 | 2013/08/02 16:04:21 | [GC 994312K->900598K(1025472K), 0.0075222 secs] INFO | jvm 1 | 2013/08/02 16:04:21 | [Fri Aug 02 16:04:21 EDT 2013] [INFO ] successfully started Name

Following File Reader Thread for the file[e:\FtpBlueCoatLogs\SG_arcsight__234786589034.log.gz] INFO | jvm 1 | 2013/08/02 16:04:21 | [GC 987894K->902358K(1006656K), 0.0091398 secs] INFO | jvm 1 | 2013/08/02 16:04:22 | [GC 989654K->915159K(1026368K), 0.0260541 secs] INFO | jvm 1 | 2013/08/02 16:04:22 | [GC 1001687K->928173K(1026240K), 0.0398470 secs] INFO | jvm 1 | 2013/08/02 16:04:23 | [Full GC INFO | jvm 1 | 2013/08/02 16:04:23 | 928173K->94713K(1026240K), 0.6081909 secs] INFO | jvm 1 | 2013/08/02 16:04:23 | [GC 181241K->107581K(1026880K), 0.0203723 secs] INFO | jvm 1 | 2013/08/02 16:04:23 | [GC 194941K->120244K(1026560K), 0.0348109 secs]

Agenda







Tuning the agent.log file

Too many logs. What can I do The default logging level for a connector is Info. Change this only when you

decide Info messages are too much to review. With some connector types the logs will rollover very fast.

Solution: Add the following lines to agent.properties file. This will take

precedence over the default setting in the agent.default.properties file. A connector restart will be required.

# Changed from 1 to 2 by me on 4/13/2013 to reduce logging messages. # 0 = Debug, 1 = Info, 2 = Warn, 3 = Error, 4 = Fatal log.channel.file.property.package.com.arcsight=2

Caution: Change this back when sending logs to support. Full detail may be needed.

Tuning the Connector Java memory settings

Java heap is divided into generations

Minor GC Only collects young generation May expand to entire heap, and become a major collection

Major GC or Full GC Collects both young generation and tenured generation

[GC 338,279K->214,397K(520,640K), 0.0037640 secs]

[Full GC 393,785K->75,819K(520,576K), 0.3965000 secs]

Tuning the Connector Java memory settings

Minor GC pause ([GC ]) Should be under 1 sec Major GC pause ([Full GC .]) Actual time depends on hardware Estimate: ~1 sec every 200 MB Heap

Working Set is defined as the memory that is in actual use and has no garbage Working set of the JVM can be found as above, immediately after a Full GC

[Full GC 932,135K->542,955K(1,036,928K), 3.9721866 secs]

Changing Memory Allocation for SmartConnector JVM

Support Knowledge document: KM1271748 Q: How do I change the memory allocation for the SmartConnector JVM? Disclaimer: The greater the size of the JVM memory, the greater is the time required

to run full garbage collection when memory runs low. In other words, there is a trade-off between the size of the JVM memory and the system performance.

1. Edit the /current/user/agent/agent.wrapper.conf file 2. Modify the following properties in agent.wrapper.conf to change the value of minimum and maximum memory used in Mb: wrapper.java.initmemory=256 wrapper.java.maxmemory=256 Example of the modified property:

wrapper.java.initmemory=512 wrapper.java.maxmemory=1024 Ref: http://support.openview.hp.com/selfsolve/document/KM1271748

http://support.openview.hp.com/selfsolve/document/KM1271748http://support.openview.hp.com/selfsolve/document/KM1271748http://support.openview.hp.com/selfsolve/document/KM1271748

Tip: Tuning the Console Java memory settings Look at your console memory in the Task Manager. Question: How to change java heap for the Console? Answer: Use these steps to change Console's Heap Size:

1. Edit \current\bin\scripts\console.bat file. Modify the following property: 2. set ARCSIGHT_JVM_OPTIONS=-Xms64m Xmx256m -XX:MaxPermSize=128m -XX:-

UseThreadPriorities -XX:+HeapDumpOnOutOfMemoryError -Dsun.java2d.noddraw=true For example to set the Console to utilize more memory set as following: 3. set ARCSIGHT_JVM_OPTIONS=-Xms512m Xmx768m -XX:MaxPermSize=128m -XX:-

UseThreadPriorities -XX:+HeapDumpOnOutOfMemoryError -Dsun.java2d.noddraw=true

Agenda







TIP: Keep SmartConnector guides handy

Download SmartConnector Guides SmartConnectorConfigGuides-6.0.4.xxxx.zip You need the guides for hours and hours of fun reading.

Did you know you have options?

See KM1270264 - Mapping Additional Data values to ArcSight fields

Lets look at an example


http://support.openview.hp.com/selfsolve/document/KM1270264http://support.openview.hp.com/selfsolve/document/KM1270264http://support.openview.hp.com/selfsolve/document/KM1270264

Administrators you need a connector installed on your workstation

Why? Connector install for:

Test Alerts RegEx LogFu Flex Agent Wizard

These tools are not in the console package Smoke test new SmartConnector upgrades on local machine

Submitting Test Alerts from your workstation

Invoke the Test Alerts script by executing the following commands: cd \current\bin arcsight agents Use CTRL-C to exit and allow SmartConnector to finish up

Running RegEx on workstation

Invoke the RegEx script by executing the following commands: cd \current\bin arcsight regex

Running LogFu on connector or manager logs

Invoke the LogFu script by executing the following commands: cd \current\logs\ ..\bin\arcsight agent logfu -a

Right-Click> Select>Show Plot/Event Window

Running the Flex Agent Wizard

Hey IT Developer; Application Security logs on internally developed applications can be simple and effective. A simple example: example_application.log

Level, Timestamp, Program File, Environment, Function, Source Employee, Destination Employee, Result, Message INFO,2013-08-03 20:18:05 EDT,C:\program

files\company.com\app\EmployeeLookup.exe,Production,Employee Lookup,56482,15695,Success,Employee general lookup successful

ALERT,2013-08-03 20:18:10 EDT,C:\program files\company.com\app\EmployeeLookup.exe,Production,Employee Salary History,56482,56865,Blocked,Unauthorized attempt to view Senior Leadership salary


\bin\arcsight flexagentwizard


Map the CSV style header to the ArcSight event fields.


Choose a the correct timestamp format.


Choose a DeviceVendor and DeviceProduct


Continue like a normal connector install.

Agenda







Command line Get Status

Support Knowledge document: KM1262493

Question: How can I view the SmartConnector status? Answer: information received by using the Console's GetStatus command, there is a command line to obtain the same however the command is un-documented and hidden. To use this command:

Navigate to the /current/bin directory on the SmartConnector host and execute the following command:

arcsight agentcommand -c status or arcsight agentcommand -c status > ../logs/connector-status.txt


http://support.openview.hp.com/selfsolve/document/KM1262493http://support.openview.hp.com/selfsolve/document/KM1262493

InstallAnywhere silent install headaches

Install a connector; cancel when it asks you for type. cd \bin runagentsetup.bat -i recorderui -g Generates an empty file. Useless! runagentsetup.bat -i recorderui Go though all the optionsWhy am I doing this? Modify the install_response_file. Why is my credentials in the clear?

# User connectordestinationnew.user=ConnectorInstaller # Password connectordestinationnew.password=C0nnector1nstaller

Replicate files for various deployments Is this going to work? On Windows ArcSight-6.0.4.6719.0-Connector-Win.exe -i silent -f install_response_file On Linux/Unix chmod 755 ArcSight-.0-Connector-Linux.bin ArcSight-6.0.4.6719.0-Connector-Linux.bin -i silent -f install_response_file


Silent install process never completes. Problem most likely during certificate acceptance. Connector is never registered. A look into the setup.log A problem occurs with Axis. [2013-08-02 19:22:31,899][INFO][default.com.arcsight.agent.nb.j][isCwsapiServerReady] AxisFault faultCode: {http://schemas.xmlsoap.org/soap/envelope/}Server.userException faultSubcode: faultString: java.net.ConnectException: Connection refused: connect

Folder never gets created. [2013-08-02 20:55:50,872][INFO ][default.com.arcsight.g.a.a.a][getFrameworkDataPath] Data directory [C:\ArcSightSmartConnectors\Syslog\current\user\logger\data] does not exist, creating new one [2013-08-02 20:55:50,872][WARN ][default.com.arcsight.common.config.c][getPropertyValue] Could not find property in configuration 'connectorappliance.relativeconfigdatadir' [2013-08-02 20:55:50,872][INFO ][default.com.arcsight.common.config.c][getPropertyValue] com.arcsight.common.config.j: An error occured in configuration. Unable to find requested property 'connectorappliance.relativeconfigdatadir'.


Could this be used for upgrades? Maybe, but why bother. Current upgrade process is very simple?

Tip: Before you uninstall; Removal of the service

Remove server service first! Connector Console Mode (Non-GUI): 1. Invoke the setup wizard by executing the following command from: cd \current\bin runagentsetup.bat or arcsight agentsetup (Advanced mode) Commandline: arcsight agentsetup -i console

2. Select: Uninstall as a service

Tip: Clean up old connector versions

/opt/arcsight/connectors/bluecoat/ |-A5717/ |-5.2.5.6395.0 |-6.0.3.7682.0 |-current/

If every thing is working you can just remove the old folders with a simple delete. Caution: Dont delete current Tip: Leave the last previous, just in case.

Summary

I suggest the following: Install connector on workstation for Test Alerts, RegEx and LogFu Perform upgrades in a workstation first Know the connector structure and key files Know your configuration files Know your logs Know your status Learn memory tuning steps Knowledge documents are there, just hard to find Keep your SmartConnector guides handy Dont be afraid to explore; its all just 1s and 0s

Tip: Download ArcSight Product stencil and icons.zip

https://protect724.arcsight.com/servlet/JiveServlet/download/15829-4266/stencils%20and%20icons.zip

https://protect724.arcsight.com/servlet/JiveServlet/download/15829-4266/stencils and icons.zip

Quick Reference

ArcSight SmartConnector Commands ARCSIGHT_HOME (Windows) C:\ArcsightSmartConnectors\myAgent\current\bin

setup runagentsetup.bat or arcsight agentsetup

setup (console mode) arcsight agentsetup -i console setup (advanced mode) arcsight agentsetup and select no at popup

startup arcsight agents or arcsight connectors

stop CTRL + C

add as a service arcsight agentsvc ?

analyze logs cd ARCSIGHT_HOME\logs ..\bin\arcsight agent logfu -a

status arcsight agentup

getstatus arcsight agentcommand -c status

SSL keytoolgui arcsight agent keytoolgui

SSL cert information arcsight agent tempca -i

FlexConnector Wizard arcsight flexagentwizard

RegEx tool arcsight regex

ARCSIGHT_HOME (Linux) /opt/arcsight/connectors/myAgent/current/bin

find smartconnectors in Linux updatedb locate agent.properties

Agenda







Please submit session feedback.

Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Security for the new reality

Are you a smart connector?About TIAA-CREFAudienceAgendaSlide Number 5Slide Number 6Slide Number 7Slide Number 8Slide Number 9Demo SmartConnector upgrades on Connector ApplianceSlide Number 11Slide Number 12Tip: Have a SmartConnector Appliance?Anatomy of a connectorSmartConnector running as a serviceLinux Tip: the locate commandAgendaSmartConnector agent.propertiesagent.properties (continued)Support Knowledge document: KM1271052SmartConnector agent.default.propertiesAgendaReading the agent.log fileReading the agent.log - Log Status agent.log (file continued)Reading the agent.out.wrapper.log fileAgendaTuning the agent.log fileTuning the Connector Java memory settingsTuning the Connector Java memory settingsChanging Memory Allocation for SmartConnector JVMTip: Tuning the Console Java memory settingsAgendaTIP: Keep SmartConnector guides handySlide Number 34Slide Number 35Slide Number 36Administrators you need a connector installed on your workstationSubmitting Test Alerts from your workstationRunning RegEx on workstationRunning LogFu on connector or manager logsRunning the Flex Agent WizardRunning the Flex Agent WizardRunning the Flex Agent WizardRunning the Flex Agent WizardRunning the Flex Agent WizardRunning the Flex Agent WizardAgendaCommand line Get StatusInstallAnywhere silent install headachesInstallAnywhere silent install headachesInstallAnywhere silent install headachesTip: Before you uninstall; Removal of the serviceTip: Clean up old connector versionsSummarySlide Number 55Quick ReferenceAgendaPlease submit session feedback.Slide Number 59

are you a smart connector? - hewlett packard enterprise · are you a smart connector? mark ulmer,...

Documents