log files slides

8/3/2019 Log Files Slides

http://slidepdf.com/reader/full/log-files-slides 1/38



The most comprehensive Oracle applications & technology content under one roof

More info at http://bit.ly/kapowelogs




Forensic process

Log files

Case files

Tools




The Forensic Process




Step One: Secure The Scene




Operating System Evidence

netstat for network issues

top or Windows Task Manager for CPU issues

iostat or vmstat for I/O issues


http://slidepdf.com/reader/full/log-files-slides 7/38The most comprehensive Oracle applications & technology content under one roof

Rolling Log Files



Cause

4-6PM2-4PM

Symptoms



Step Two: Investigate The Scene



Don’t.

Search.

The.Log.

Files.



‘Error’ versus ‘Warning’

‘Failing’ versus ‘Failed’



Step Three: Gather And CorrelateEvidence



Step Four: Build A Hypothesis



1) Secure the scene

2) Investigate the scene

3) Gather and correlate evidence

4) Build a hypothesis



Forensic process

Log files

Case files

Tools



AdminServer

managedServer2

managedServer1

WebLogic Server Domain

Java

processes



HTTP Access Logs



192.168.5.6 - - [19/Nov/2010:13:34:49 +0800] "POST /AccountServices/ProxyServices/AccountServices HTTP/1.1" 200 29487

192.168.5.6 - - [19/Nov/2010:13:34:49 +0800] "POST /WarehousingServices/ProxyServices/RequestOrderDetails HTTP/1.1" 200 1167

192.168.5.6 - - [19/Nov/2010:13:34:49 +0800]

"POST /WarehousingServices/ProxyServices/RequestOrderDetails HTTP/1.1“

200 1167

Remote host

rfc931

authuser

date

request

status bytes



ELF = Extended Logging Format




Extended Logging Format Fields

Common format fieldsdate

time

bytes

sc-status

Network fieldsc-ip

s-ip

c-dnss-dns

Request fieldscs-method

cs-uri

cs-uri-stem

cs-uri-query

The Good Stuff cs-comment

time-taken

custom




Server log files




####<2/08/2011 12:49:35 AM EST> <Notice> <Server> <brother-eye> <AdminServer> <[ACTIVE] ExecuteThread: '0' for queue:

'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1312210175933> <BEA-002613> <Channel "Default" is now listening on

10.0.2.15:7001 for protocols iiop, t3, ldap, snmp, http.>

####<2/08/2011 12:49:35 AM EST> <Notice> <WebLogicServer> <brother-eye> <AdminServer> <[ACTIVE] ExecuteThread: '0' for queue:

'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1312210175933> <BEA-000331> <Started WebLogic Admin Server

"AdminServer" for domain "example1030Domain" running in Development Mode>

<2/08/2011 12:49:35 AM EST> <Notice> <WebLogicServer> <brother-eye>

Timestamp Severity Subsystem Machine

<AdminServer> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default

(self-tuning)'>

Server Thread ID

<<WLS Kernel>> <> <> <1312210175933> <BEA-002613> <Channel "Default" is

User Txn ID Diagn. Time (msecs) Message ID Text




Debug flags




HTTP: weblogic.servlet.DebugHttp

SSL: default.DebugSSL

JDBC:weblogic.jdbc.sql.DebugJDBCSQL




Oracle Service Bus tracing




JMS Message Logs

SOA S it Di ti L




SOA Suite Diagnostic Logs




Forensic process

Log files

Case files

Tools




Case File #1

An Unbalanced Load




Load balancer

Sun Reverse

Proxy

Sun Reverse

Proxy

WebLogic Server

WebLogic Server




cat access.log* | awk ‘{ print $x }’ | sort | uniq

(where x = position of the cookie in the log file)




Case File #2

Fear Of Commitment




Oracle Service Bus

Tuxedo




Forensic process

Log files

Case files

Tools




Tools

Editors

The Gun

vi

Querying data

find

grepsed

awk

tail

Analysis

Excel

RSplunk



@kapowe

kevinpowe

kapowe

log files slides

Documents