Upload File

sydney python presentation (october 2010) - splunk

14
Splunk and Python Sydney Python October 2010 Kelvin Nicholson

Upload: kelvin-nicholson

Post on 27-Jan-2015

118 views

Category:

Technology


0 download

Report

Tags:

DESCRIPTION

This was a presentation I gave about Splunk to the Sydney Python group in October 2010. I talked in depth about modifying Splunk for interesting added functionality.

TRANSCRIPT

Page 1: Sydney Python Presentation (October 2010) - Splunk

Splunk and Python

Sydney Python October 2010

Kelvin Nicholson

Page 2: Sydney Python Presentation (October 2010) - Splunk

What is Splunk?

“Splunk is the world’s leading software used to monitor, report and analyze live streaming IT data as well as terabytes of historical data – located on-premises or in the cloud.” -Splunk.com

“Splunk is like google for log files.” -Kelvin

Page 3: Sydney Python Presentation (October 2010) - Splunk

Installing Splunk (on Ubuntu)

$ sudo dpkg -i splunk-4.1.5-85165-linux-2.6-intel.deb$ sudo splunk enable boot-start$ sudo /etc/init.d/splunk start

Page 4: Sydney Python Presentation (October 2010) - Splunk

Splunk Welcome Screen

Page 5: Sydney Python Presentation (October 2010) - Splunk

Configuring Splunk●Configure Splunk to allow syslog traffic●Configure devices to send syslog to Splunk

○ Linux (syslog-ng) destination loghost { udp("192.168.83.11" port (514)); }; log { source(s_all); destination(splunk); };

●Cisco IOS no logging console no logging monitor logging 192.168.83.11

● OSSEC <syslog_output> <server>192.168.83.11</server> <port>8514</port> </syslog_output>

Page 6: Sydney Python Presentation (October 2010) - Splunk

Splunk Search Screen

Page 7: Sydney Python Presentation (October 2010) - Splunk

Why I Like Splunk (Abridged)

●Dashboards of Search terms■ Security alerts “login failed for”■ STP network issues (“LEARNING AND FORWARDING”■ Duplex mismatches■ Wildcard searches, e.g. “-server2k3-”

●My “WTF” filter (easy filter building)●Beautiful trending (“cold start” AND “switch01”)

Page 8: Sydney Python Presentation (October 2010) - Splunk

Splunk Simple Filtering

Page 9: Sydney Python Presentation (October 2010) - Splunk

Extending Splunk with Python

●REST API. (Search only)●Custom search command. (iplocation)●Configuring scripted alerts. (tweet X alert)●Directly to backend using Splunk's built-in

modules. (Full module access)

Page 10: Sydney Python Presentation (October 2010) - Splunk

Accessing Splunk Datastore>>> import splunk.auth, splunk.search>>> key = splunk.auth.getSessionKey('admin','changeme')>>> my_job = splunk.search.dispatch('search sypy', namespace='search')>>> event_list = []>>> for event in my_job.events:... event_list.append(event.fields)... >>> print event_list

kelvinn@splunk:/opt/splunk/bin$ ./splunk cmd python

[{'_si': splunk,main, 'index': main, 'sourcetype': syslog, 'source': udp:514, '_kv': 1, 'splunk_server': splunk, '_time': 2010-10-06T19:40:37+1100, 'host': 192.168.83.5, '_sourcetype': syslog, '_raw': Oct 6 19:40:37 192.168.83.5 Oct 6 19:40:38 mini kelvinn: hello SyPy, hope you are doing well., '_serial': 0, '_cd': 0:275}, {'_si': splunk,main, 'index': main, 'sourcetype': syslog, 'source': udp:514, '_kv': 1, 'splunk_server': splunk, '_time': 2010-10-06T19:39:33+1100, 'host': 192.168.83.5, '_sourcetype': syslog, '_raw': Oct 6 19:39:33 192.168.83.5 Oct 6 19:39:34 mini kelvinn: sypy, '_serial': 1, '_cd': 0:251}]

>>> event_list[0]['_raw']Oct 6 19:40:37 192.168.83.5 Oct 6 19:40:38 mini kelvinn: hello SyPy, hope you are doing well.

Page 11: Sydney Python Presentation (October 2010) - Splunk

Splunk Architecture

CherryPy built-in, sweet. What can we do with that?

Page 12: Sydney Python Presentation (October 2010) - Splunk

Built-in CherryPy Funkelvinn@splunk:/opt$ cat splunktest.py import cherrypyimport splunk.auth, splunk.search

def get_splunk_data():key = splunk.auth.getSessionKey('admin','changeme') # replace with your credentialsmy_job = splunk.search.dispatch('search sypy', namespace='search', earliest_time='-24h')

event_list = []for event in my_job.events:event_list.append(event.raw)return event_listclass HelloWorld:def index(self):splunk_list = get_splunk_data()return str(splunk_list)index.exposed = True

cherrypy.config.update({'server.socket_host': '0.0.0.0','server.socket_port': 9999,})cherrypy.quickstart(HelloWorld())kelvinn@splunk:/opt$ /opt/splunk/bin/splunk cmd python /opt/splunktest.pyP.S. I'm not a CherryPy expert, but it looks pretty fun.

Page 13: Sydney Python Presentation (October 2010) - Splunk

View CherryPy Page

Page 14: Sydney Python Presentation (October 2010) - Splunk

Resources + ThanksSplunk introduction:

http://www.splunk.com/base/Documentation/4.1.5/Installation/Splunksarchitectureandwhatgetsinstalled

Splunk REST Search (with Python httplib example):

http://www.splunk.com/base/Documentation/4.1.5/Developer/RESTCreateSearch

Custom search command (iplocation):

http://www.splunk.com/base/Documentation/latest/SearchReference/Customsearchiplocation

How to write custom alerts:

http://www.splunk.com/base/Documentation/4.1.5/Admin/Configurescriptedalerts

Using Splunk's built-in Python modules:

http://answers.splunk.com/questions/14/can-i-use-splunks-built-in-python-sdk-in-my-own-scripts

Some information about Splunk's Python SDK:

http://www.splunk.com/base/Documentation/4.1.5/Developer/PySDK

Thanks.

http://www.splunk.com/base/Documentation/4.1.5/Installation/Splunksarchitectureandwhatgetsinstalled
http://www.splunk.com/base/Documentation/4.1.5/Installation/Splunksarchitectureandwhatgetsinstalled
http://www.splunk.com/base/Documentation/4.1.5/Developer/RESTCreateSearch
http://www.splunk.com/base/Documentation/4.1.5/Developer/RESTCreateSearch
http://www.splunk.com/base/Documentation/latest/SearchReference/Customsearchiplocation
http://www.splunk.com/base/Documentation/latest/SearchReference/Customsearchiplocation
http://www.splunk.com/base/Documentation/4.1.5/Admin/Configurescriptedalerts
http://www.splunk.com/base/Documentation/4.1.5/Admin/Configurescriptedalerts
http://answers.splunk.com/questions/14/can-i-use-splunks-built-in-python-sdk-in-my-own-scripts
http://answers.splunk.com/questions/14/can-i-use-splunks-built-in-python-sdk-in-my-own-scripts
http://www.splunk.com/base/Documentation/4.1.5/Developer/PySDK
http://www.splunk.com/base/Documentation/4.1.5/Developer/PySDK

Netfilter Iptables for Splunk Documentation - Read the Docs · Netfilter Iptables for Splunk Documentation, Release 0 Splunk Answers Splunk has a strong community of active users

Splunk - Splunk for Industrial Data and the Internet of Things

Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

Splunk For IT Operational Intelligenceuploads.integrity360.com/1425903243-Splunk-for-IT-Operations.pdf · • Splunk Apps –Splunk App for VMware –Splunk Apps for Citrix & Hyper

Agency Chargeback Models to Enable Enterprise Splunk ......Workshops: Get Splunk Hands -on Experience Attend a Splunk Workshop Upcoming Schedule December 1: Introduction to Splunk

Splunk Review - University of Birmingham · Ref: Splunk Review, Dec 2014 Mohammad Rameez Shafsad [email protected] Indexing 2.2.1 Searching using splunk Splunk uses

Lenovo Network Advisor for Splunk · 6 Lenovo Network Advisor for Splunk Deployment and User Guide Splunk Components A Splunk solution consists of the following components: Indexer

SplunkingYour Mobile Apps - .conf19 | Splunk · Introducing Splunk for Mobile Intelligence (Splunk MINT) Deploying Splunk MINT – Intro to SDKs – Getting Started Using Splunk MINT

SPLUNK® ENTERPRISE - CDWwebobjects.cdw.com/.../Splunk/Product-Brief-Splunk-Enterprise-The... · Splunk Enterprise collects data from any source, including logs, clickstreams, sensors,

Splunk Cloud Product Briefwinncom.com.ua/wp-content/uploads/2018/07/splunk-Log_Flow.pdf · Extend Benefits With Splunk Apps Splunk Cloud includes support for premium solutions such

ReversingLabs Explainable Threat Intelligence Enriches ... Data Sheets/RL-Splunk...sent to Splunk. • In Splunk, the TitaniumScale report is correlated with other available Splunk

Redapt @ splunk .conf 2013 splunk in the hyperscale private cloud

Fortscale Splunk Integrationinfo.fortscale.com/hubfs/UEBA Content/Fortscale Splunk Integration.pdf · Fortscale Splunk Integration ... Fortscale digests access and authentication

Splunk conf2014 - Onboarding Data Into Splunk

Sydney Python Presentation (Feb 2010) - Tracking Large Metallic Objects / Google App Engine

Netfilter Iptables for Splunk Documentation - Read … · Netfilter Iptables for Splunk Documentation, Release 0 Splunk Answers Splunk has a strong community of active users and

Splunk conf2014 - Splunk for Data Science

SPLUNK Capabilities Overview - GTRI · Overview GTRI is the only ELITE Splunk integrator with certified Splunk architects and engineers on staff. In addition to ... SPLUNK-Overview

Rivium Splunk Windows · o Splunk Enterprise Security * o uberAgent* o Splunk App for Web Analytics Common Splunk Apps & Add-ons 15 SplunkingWIndows o Splunk Add-on for Microsoft

Splunk user group - automating Splunk with Ansible

データ分析プラットフォーム「Splunk」 - Panasonicdl.it-sol.jpn.panasonic.com/data/splunk/splunk.pdfTitle データ分析プラットフォーム「Splunk」 Author パナソニック

Tenable and Splunk Integration · Splunk Splunkreceivesvulnerabilitydatacollectedby SecurityCenter Nessus NessusHost Scans,Nes-susPlugins Splunk SplunkreceivesvulnerabilitydatacollectedbyNes-

Splunk conf2014 - Splunk Monitoring - New Native Tools for Monitoring your Splunk Deployment

Splunk For IT Operational Intelligence · • Splunk Apps –Splunk App for VMware –Splunk Apps for Citrix & Hyper V –Splunk App for Microsoft Exchange –Splunk App for Active

Our experiences throughout the 3 year journey · Splunk – Hardware is now available for Splunk expansion • Splunk begins to fill monitoring gaps, acts as “glue” • Splunk

Splunk Conf2010: Corporate Express presents Splunk with SAP

Referent / Redner Benjamin Tiggemann · 2015-07-08 · Splunk for Exchange Splunk for Enterprise Security* Splunk for Vmware Splunk for PCI Compliance* Splunk for Windows Splunk for

Splunk Inc. Splunk 4.1.7 Security Target - Common Criteria · 1.1 ST Reference ... 7.1.3.2 FDP_ACF.1 Access control functions ... ST Title: Splunk Inc. Splunk 4.1.7 Security Target

stoQ’ing your Splunk - SANS · PDF filestoQ’ingyour Splunk Ryan Kovar, Splunk Marcus LaFerrera, PUNCH SANS DFIR 2016

SplunkLive Sydney Scaling and best practice for Splunk on premise and in the cloud

Admission Control using Splunk Enterprise Deployment Guide€¦ · Configuring Splunk Enterprise ... For more information on Splunk configuration, see Splunk documentation. Troubleshooting

Splunk .conf2011: Splunk for Fraud and Forensics at Intuit

Monitoring Docker Containers with Splunk - Splunk .conf · Splunk and Docker – At A Glance Visibility in your Container Environments Delivering Splunk as Containers Monitoring for

NetFlow Collection Analysis: NFCAPD Python Splunk | GIAC

Splunk Spark Integration - GitHub Pageslitaotao.github.io/files/4. splunk_spark.pdf · Splunk Spark Integration ... • Splunk"products:""! Splunk"Enterprise"! ... splunk_spark Created