blackhat workshop

OSSEC WorkshopWim Remes - Xavier Mertens

BH EU 2011

Thursday 17 March 2011

About Us

• Wim

• works for EY Belgium

• Security Consultant

• Eurotrash

• InfoSec Mentors

• Brucon

• Xavier

• Senior Security Consultant for a Belgium company

• Security Blogger

•


Technical Breakdown


• Mix of OS / Application / Protocols

• Thousands of events to process

• Multiple consoles/tools

• Keep Security at the highest level(“CIA” principle)

Technical Issues


Find the Differences...

• Aug 27 14:33:01 macosx ipfw: 12190 Deny TCP 192.168.13.1:2060 192.168.13.104:5000 in via en1

• %PIX-3-313001: Denied ICMP type=11, code=0 from 192.168.30.2 on interface 2


Economic Issues

• “Time is Money” (24x7, no downtime)

• Reduced staff & budget

• Happy shareholders

• This costs $$$ and HH:MM! (Commercial as well as Free!)


Legal Issues

• Compliance requirements (by “group” or by business)

• Local laws (retention, data protection)

• Due diligence & due care


Challenges

• Creation and archiving of log files (centralized)

• Analyze (Normalization)

• Follow-up

• Reporting


Layers Model

Log Collection

Normalization

Storage

Search

Correlation

Reporting


OSSEC in a Nutshell“Because everybody must take care of logs”


• OSSEC is an free HIDS• Features

• Log Analysis / File Integrity Checks

• Policy Monitoring

• Rootkit Detection

• Actions (Alerts / Active Response)

• Open to 3rd party products

Core Features


OSSEC Position

Log ManagementSolutions

SIEMSolutions

Focus on Logs Focus on Security

OSSEC


OSSEC cannot...

• Detect access to files (or based on info provided by the OS)

• Use proprietary protocols > You have to convert them to Syslog (ex: CheckPoint)

• Display nice graphs

• OSSEC is just a (dumb) tool!


It’s not a product...(c) Bruce

• Problems? Results!

• Proof of Concept with limited scope

• Tests procedure from A to Z

• Procedures! (yeah, boring)


Starter’s Kit

• A Linux box

• Enough Storage

• Some UNIX/networking knowledge

• Script-Fu can be helpfull

• Free time!


• Architecture

• Server

• Agents (UNIX & Windows)

• DB (optional)

• 3rd Party Products (optional)

Architecture


Software Components

Server Agentlogcollector x x

agentd (x) xexecd x x

syscheckd x xanalysisd x

maild xremoted xmonitord xreportd xcsyslogd x


Supported Log Formats

• UNIX & tools

• FTP / SMTP / HTTP servers

• Firewalls

• DB’s

• Security Tools

• Commercial (CP, VMware, Bluecoat, ...)

• Almost anything (custom decoders)


Decoded Variables

location• hostname• log_tag• srcip, dstip• srcport, dstport• protocol• action• user, dstuser• id

• command• url• data


Server Installation

• Harden Your Linux Server

• Allow traffic to UDP/1514

• ./install.sh && Answer questions

• ./manage-agents && Create keys


$HOME Sweet $HOME

• ossec.conf

• local_rules.conf

• decoder.xml

• ossec-logtest


Agents Phone $HOME

• Both directions UDP/1514!

• Tools

• manage_agents

• list_agents

• agent_control


Centralized Management

• $OSSECHOME/etc/shared/agent.conf

• Setup config blocks as ossec.conf <agent_config name=”myagent”> <localfile> <location>/var/log/mylog</location> <log_format>syslog</log_format> </localfile> </agent_config


Reporting

• Simple reporting is provided thru ossec-reportd: -f <filter> <value>

-r <filter> <value>

Example: -f group authentication failed -f level 10 -f group authentication -r user srcip


Reporting (cont)

• Top-20 Offending IP addresses

• Top-20 Offending users

• Top-20 Suspicious alerts

• Top-20 Triggered alerts


Log Archives

• Enable with the following keyword(default off): <logall>on</logall>

• MD5/SHA1 for integrity

• Raw event is stored! (evidences)


Alerts Post Analysis

• OSSEC has a WUI but outdated (IMHO)

• Alternatives

• Picviz

• Prelude

• Splunk or LaaS (Loggly) <syslog_output> <server>127.0.0.1</server> <port>10002</port> </syslog_output>


Key Design&

Implementation Issues


Time Synchronization

• Use NTP to synchronize your devices

• Mandatory to investigate security incidents


Access Raw Data

• Safe & reliable collection of Syslog flows

• Access to local files (agents)


UDP 1514

• OSSEC adds confidentiality (packets are encrypted) but still relies on UDP

• No caching or heart-beat mechanism


High Availability

• Full Virtual IP + storage sync (Active/Passive)

• Multiple Servers (Failover) # ossec.conf <client> <server-ip>192.168.0.10</server-ip> <server-ip>192.168.10.10</server-ip> </client>

# internal_options.conf remoted.verify_msg_id=0


Long Term Retention

• $OSSECHOME/logs/archives/YYYY/MMM

• Could fill your filesystem very quickly!

• Procedure must be implemented for long term retention (ex: NAS, DVDs)


Agents Mass-Deployment

• ossec-batch-manager.pl (contrib)

• Deployment tools

• cfengine (UNIX)

• Active Directory (Windows)

• New!!

• Server :

• Client :# /var/ossec/bin/ossec-authd -p 1515 >/dev/null 2>&1 &

# /var/ossec/bin/agent-auth -m 192.168.1.1 -p 1515


Building/CustomizingOSSEC rules


Basics• $OSSECHOME/rules

• local_rules.xml 1

2 3

4 5 6


Basicsstep 1 : decoder.xml

<decoder name="sshd"> <program_name>^sshd</program_name></decoder>

<decoder name="sshd-success"> <parent>sshd</parent> <prematch>^Accepted</prematch> <regex offset="after_prematch">^ \S+ for (\S+) from (\S+) port </regex> <order>user, srcip</order> <fts>name, user, location</fts></decoder>


Basicsstep 1 : decoder.xml

<decoder name="sshd"> <program_name>^sshd</program_name></decoder>

<decoder name="ssh-denied"> <parent>sshd</parent> <prematch>^User \S+ from </prematch> <regex offset="after_parent">^User (\S+) from (\S+) </regex> <order>user, srcip</order></decoder>


Basicsstep 2 : /var/ossec/sshd_rules.xml

<rule id="5700" level="0" noalert="1"> <decoded_as>sshd</decoded_as> <description>SSHD messages grouped.</description></rule> <rule id="5716" level="5"> <if_sid>5700</if_sid> <match>^Failed|^error: PAM: Authentication</match> <description>SSHD authentication failed.</description> <group>authentication_failed,</group></rule><rule id="5720" level="10" frequency="6"> <if_matched_sid>5716</if_matched_sid> <same_source_ip /> <description>Multiple SSHD authentication failures.</description> <group>authentication_failures,</group> </rule>

5700

5716

5720


Basics

<rule id="100001" level="0"> <if_sid>5711</if_sid> <srcip>1.1.1.1</srcip> <description>Example of rule that will ignore sshd </description> <description>failed logins from IP 1.1.1.1.</description></rule>

step 3 : $OSSECHOME/rules/local_rules.xml

$OSSECHOME/bin/ossec-logtest


Hands-on


Lab Environment

• ssh student@yourhost (Pass: 0SSEC4ever)

• sudo -s

• Stuff in $HOME/files/

• Live Syslog feed received in /var/log/

• Sendmail available

• Do NOT abuse!


Exercice #1

• Install OSSEC (stand-alone)

• Start collecting events

• Play with configuration files

• Send notifications via e-mail


Exercise #2

• Generate an (email) alert when accesses to Facebook are detected


Solution #2

• In $OSSECHOME/rules/local_rules.xml:  <rule id=”100030” level=”10”> <match>facebook.com</match> <description>Access to Facebook detected!</description> </rule>

• Restart OSSEC

Exercice #3

• Monitor (decode) an unknown file format: /var/log/application.log

• Report activity for the user ‘admin’

• Tip: Use ossec-logtest


Solution #3

• Log format: Mar 10 23:36:43 foo application[4583]: john created /data/report134.ppt

• In $OSSECHOME/etc/decoder.xml: <decoder name=”newapp”> <program_name>application</program_name> </decoder> <decoder name=”newapp-event”> <parent>newapp</parent> <regex>^(\S+)</regex> <order>user</order> </decoder>


Solution #3 (cont)

• In $OSSECHOME/etc/ossec.conf: <localfile> <log_format>syslog</format> <location>/var/log/application.log</location> </localfile>


Solution #3 (cont)

• In $OSSECHOME/rules/local_rules.xml: <rule id=”100040” level=”0”> <decoded_as>newapp</decoded_as> <description>New Application Event</description> </rule> <rule id=”100041” level=”10”> <if_sid>100040</if_sid> <user>admin</user> <description>User admin activity detected</description> </rule>

• Restart OSSEC


Exercice #4

• Suspicious access detection

• Detect SSH access from Belgium

• Tips

• Use an Active-Response script

• GeoIP API in $HOME/files/geoip


Solution #4

• Install the GeoIP RPM

• Copy the new Active-Response (geoip.sh) script in $OSSECHOME/active-response/bin

• Review the script content


Solution #4 (cont)

• Configure the Active-Response script in $OSSECHOME/etc/ossec.conf <command> <name>geoip-lookup</name> <executable>geoip.sh</executable> <expect>srcip</expect> </command>


Solution #4 (cont)

• Find the right rules to attach the Active-Response to (ex: #5501 - Login session opened)

• Link the Active-Response to the rule: <active-response> <command>geoip-lookup</command> <location>server</location> <rules_id>5501</rules_id> <active_response>

• Restart OSSEC


Solution #4 (cont)• Monitor the new logfile

<localfile> <location>/var/log/geoip.log</location> <log_format>syslog</log_format> </localfile>

• Create a new rule <rule id=”100100” level=”10”> <regex>Detected \S+ from BE, Belgium</regex> <description>Suspicious login from Belgium</description> </rule>

• Restart OSSEC and watch alerts.log


Other Examples

• MySQL database integrity audit

• USB-stick detection on Windows

• Rogue access detection (using geo-localization)

• Mapping data on Google Maps

• Temporary lookup tables


Happy Logging!

xavier (at) rootshell (dot) bewremes (at) gmail (dot) com


blackhat workshop

Documents

home ossec

ossec reportd

core features ossec

nice graphs ossec

alerts post analysis

agents agent

conf local

log archives