1 abstracting the content of system call traces waseem fadel abdelwahab hamou-lhadj department of...
Post on 29-Dec-2015
213 Views
Preview:
TRANSCRIPT
1
Abstracting the Content of System
Call Traces
Waseem FadelAbdelwahab Hamou-Lhadj
Department of Electrical and Computer EngineeringConcordia UniversityMontréal, QC, Canada{w_fadel, abdelw}@ece.concordia.ca
2
Objective of the Trace Abstraction and Analysis Track (Reminder)
Develop trace abstraction techniques and tools to facilitate the understanding and
analysis of the content of large event-based system call traces
3
Benefits Help users understand the behavioural aspects of a
system Allow automated comparison of multiple traces
based on the system behaviour and not a mere set of events
Monitor the system performance to detect service degradation
Ensure that subsequent revisions of a software system have not introduced programming errors
Compare traces of redundant servers, performing the same work in order to detect any malfunction possibly caused by a security breach
4
Proposed Approach
Linux KernelDocumentation
Sample LTTng Traces
Expert Knowledge
Trace Generated from
Linux Kernel(LTTng events)
Trace Abstraction Algorithm
Based on Pattern Matching and
Filtering of Noise
High-Level Trace
Pattern Library
5
Pattern Library
We built a pattern library that contains several patterns that represent key Linux kernel operations File, socket and process management operations
The patterns are modeled as state machines States represent system modes (user_mode, sys_call
mode, etc.) Events consist of LTTng events
6
Patterns we have so far
We completed the pattern library to include the following operations: File Management (Open, Read, Write, Seek,
Access, File Manipulation with fcntl, Stat, Close) Socket Management for both TCP and UDP
(Create, Connect, Bind, Listen, Accept, Send, Receive, Close)
Process Management (Execution with exec and execve, Exit, Fork, Clone, Get Resource Limit, Get Time of The Day, New UName, Unlink, Read Link)
7
Filtering of Trace Noise
We define noise in an LTTng trace as any event associated with memory management, page faults, and interrupts Are dependent on a specific kernel version Can occur anywhere in the trace and in any order Are treated similarly to the way utilities have been treated
in related work
Associated events are treated as a set i.e. order of occurrence of detailed events is ignored
8
Validation of Patterns
The patterns have been validated by Pierre-Marc Fournier and Mathieu Desnoyers from École Polytechnique de Montréal
Regular meetings with them have also helped in the process of understanding the markers, the system calls, and the main modes of execution
Both users agreed with the way we defined noise found in traces
9
The Linux Kernel Trace Abstraction Tool
The tool takes as input a trace generated from LTTng tracer
It applies our approach to that trace Outputs the trace in its abstracted format It has been developed in Java as an Eclipse
plug-in
10
The Linux Kernel Trace Abstraction Tool (cont.) The tool was designed to accept patterns defined as external XML files
<?xml version="1.0" encoding="ISO-8859-1"?><pattern name="Sample Pattern" type="HighLevelSampleConstrcut"
noise="false"> <event name="syscall_entry" syscall_name="sys_sample"
order="1" prev_state="IGNORE"> <current_state>SYSCALL_SAMPLE</current_state> </event>
<event name="sample" order="2" prev_state="SYSCALL_SAMPLE"> <current_state>SYSCALL_SAMPLED</current_state> </event>
<event name="syscall_exit" order="LAST" prev_state="SYSCALL_SAMPLED"> <current_state>USER_MODE_SAMPLED</current_state> </event>
</pattern>
11
The Linux Kernel Trace Abstraction Tool (cont.)
12
The Linux Kernel Trace Abstraction Tool (cont.)
The tool consists of three parts: The top part, through which the developer can
provide the tool with the required information The middle part, displays the trace resulting
from the abstraction process The bottom part, displays the information
related to the original and abstracted traces The following slides show the architecture
and the class diagram
13
Architecture
Partitioned both horizontally and vertically Easy to extend (to add new system calls
patterns or even patterns for different calls) Easy to maintain (to modify existing patterns
or the design itself)
14
Horizontal Partitioning
Horizontal partitioning is performed by defining the main domains of the system
15
Vertical Partitioning
16
Vertical Partitioning (cont.)
Divide the system into different layers Define the interfaces between layers Presentation layer can be developed without
affecting lower layers (Multiple GUIs can be provided for the same data)
Components from different layers can be designed, implemented and maintained independently
17Class Diagram
18
Class Diagram (cont.)
Adding new patterns and high-level constructs can easily be done by sub-classing the appropriate classes and interfaces
Multiple implementations representing different trace formats can be applied using the same interfaces
High level constructs are easy to further abstract
19
Case Studies
We applied our approach to large traces generated while running different processes
One process was the java virtual machine which was running a distributed file server and a client
Another process was the eclipse framework Third process was gedit Fourth process was GIMP image editor The final process was firefox
20
Case Studies (cont.)
ProcessInitial size
Size after Abstraction
Compression Ratio
Eclipse122698532590273%
GIMP84757522977873%
Firefox64671025728260%
Gedit1861679048251%
JVM47271303393%
21
Case Studies (cont.)
SEQ(1) Socket Create: family = 2, type = 2, protocol = 0, sock = 0xd563d340, ret = 8SEQ(1) Socket Connect: fd = 8, uservaddr = 0x80569c8, addrlen = 28, ret = 0SEQ(2) File Stat: ip = 0xb7f78430, syscall_id = 221 [sys_fcntl64+0x0/0xb0]SEQ(1) Get Time of Day: ip = 0xb7f78430, syscall_id = 78 [sys_gettimeofday+0x0/0x80]SEQ(1) Unknown Event: Event name: pollfd, Params: fd = 8SEQ(1) Unknown Event: Syscall name: sys_poll, Params: ip = 0xb7f78430, syscall_id = 168
[sys_poll+0x0/0xc0]SEQ(1) Socket Send: call = 9, a0 = 8SEQ(4) Unknown Event: Event name: pollfd, Params: fd = 8SEQ(1) Process Schedule: prev_pid = 0, next_pid = 23566, prev_state = 0
…
SEQ(1) Unknown Event: Syscall name: sys_futex, Params: ip = 0xb7f78430, syscall_id = 240 [sys_futex+0x0/0x130]
SEQ(1) Unknown Event: Syscall name: sys_clock_gettime, Params: ip = 0xb7f78430, syscall_id = 265 [sys_clock_gettime+0x0/0xa0]
SEQ(1) Get Time of Day: ip = 0xb7f78430, syscall_id = 78 [sys_gettimeofday+0x0/0x80]SEQ(1) File Access: ip = 0xb7f78430, syscall_id = 33 [sys_access+0x0/0x30]SEQ(1) File Write: fd = 19SEQ(1) Socket Receive: call = 10, a0 = 19SEQ(1) Process Exit: Process Exit: pid = 23580, cpu_id = 1, state = 1, Send Signal: pid = 23610, cpu_id
= 1, state = 1, Sched Try Wakeup: pid = 23972, cpu_id = 1, state = 1
A snapshot of the traces resulting from the abstraction of the JVM process
22
Remaining Challenges
Continuous improvement of the pattern library Defining additional patterns Dealing with new LTTng events
Using higher level constructs to further abstract the resulting traces
Improving the algorithm in terms of performance
Embedding the tool with the LTTv
23
Conclusion
We introduced techniques to abstract execution traces resulting from the Linux kernel
Our approach is based on building a pattern library that consists of patterns of the most common operations in Linux
We also defined noise patterns that result from memory management operations and page faults
We introduced an algorithm to abstract the system call traces by using the pattern library
We applied our techniques to traces generated from several processes
24
Pattern Library
25
File Management: Open & Close
SYSCALL_OPEN
SYSCALL_FILE_OPENED
USER_MODE_FILE_OPENED
syscall_entry(sys_open)
open
syscall_exit
Within a Process
SYSCALL_CLOSE
USER_MODE_FILE_CLSOED
syscall_entry(sys_close)
close
UESR_MODE_FILE_OPENED
SYSCALL_CLOSED
syscall_exit
Open File Close File
26
File Management: Read & Write
SYSCALL_READ
SYSCALL_DATA_READ
syscall_entry(sys_read)
read syscall_exit
UESR_MODE_FILE_OPENED
SYSCALL_WRITE
SYSCALL_DATA_WRITTEN
syscall_entry(sys_write)
syscall_exit
UESR_MODE_FILE_OPENED
write
Write to File Read from File
27
Socket Management (1)
SYSCALL_SOCKETCALL
SYSCALL_SOCKET_CREATION
USER_MODE_SOCKET_CREATED
syscall_entry(sys_socketcall)
socket_call (call = 1)
socket_create
Within a Process
SYSCALL_SOCKET_CREATED
syscall_exit
SYSCALL_SOCKETCALL
SYSCALL_SOCKET_BINDING
USER_MODE_SOCKET_BOUND
syscall_entry(sys_socketcall)
socket_call (call = 2)
socket_bind
Within a Process
SYSCALL_SOCKET_BOUND
syscall_exit
SYSCALL_SOCKETCALL
SYSCALL_SOCKET_LISTENING
USER_MODE_SOCKET_LISTENING
syscall_entry(sys_socketcall)
socket_call (call = 4)
socket_listen
Within a Process
SYSCALL_SOCKET_STARTED_LISTENING
syscall_exit
Create Bind Listen
28
Socket Management (2)
SYSCALL_SOCKETCALL
SYSCALL_SOCKET_CONNECTING
USER_MODE_SOCKET_CONNECTED
syscall_entry(sys_socketcall)
socket_call (call = 3)
socket_connect
Within a Process
SYSCALL_SOCKET_CONNECT
syscall_exit
SYSCALL_SOCKETCALL
SYSCALL_SOCKET_SEND
USER_MODE_SOCKET_SEND
syscall_entry(sys_socketcall)
socket_call (call=9 || call=11)
Within a Process
SYSCALL_DEV_XMIT
syscall_exit [napi_completed=true]
SYS_CALL_NAPI_SCHEDULE
napi_schedule
SYSCALL_NAPI_POLL
napi_poll
SYSCALL_DEV_RECEIVE
dev_receive
entry/napi_comleted=true
SYSCALL_NAPI_COMPLETE
napi_complete
SYSCALL_TIMER_SET
timer_set
timer_set
dev_xmit
Connect Receive Send
SYSCALL_SOCKETCALL
SYSCALL_SOCKET_RECEIVE
USER_MODE_SOCKET_RECEIVE
syscall_entry(sys_socketcall)
socket_call (call=10 || call=12)
dev_xmit
Within a Process
SYSCALL_DEV_XMIT
syscall_exit
SYS_CALL_NAPI_SCHEDULE
napi_schedule
SYSCALL_NAPI_POLL
napi_poll
SYSCALL_DEV_RECEIVE
dev_receive
SYSCALL_NAPI_COMPLETE
napi_complete
SYSCALL_TIMER_SET
timer_set
29
Socket Management (3)
entry/isBound=true
BindConnect
entry/isListening=true
Listen
Accept
Send
Receive
socket_call (call = 9)
socket_call (call = 9)
socket_call (call = 10)
socket_call (call = 10)
Close
SYSCALL_SOCKETCALL
sys_socketcall
socket_call (call = 1)
sys_socketcall
sys_socketcall
sys_close
socket_call (call = 9)
socket_call (call = 10)
sys_close
sys_close
sys_close
sys_close sys_close
Create
Within a Process
sys_socketcall
socket_call (call = 2)
socket_call (call = 3)
SYSCALL_SOCKETCALL [SOCKET_CREATED]
sys_close
socket_call (call = 4) [isBound==true] socket_call (call = 5) [isListening==true]
TCP Connection
30
Process Management (1)
Cloning/Forking
Within a Process
ptregs_clone
File Management
TCP SocketsWithin a Child Process
sys_waitpid
SYSCALL_WAIT
SYSCALL_WAITING
process_wait
UDP Sockets
Execution with execve
ptregs_execve
Exit Process
sys_exit_group
ptregs_clone
sched_schedule
SYSCALL_SCHED_SCHEDULE
USER_MDOE_SCHEDULED
syscall_exit
ptregs_clone
31
Process Management (2)
SYSCALL_EXEC
USER_MODE_EXEC
syscall_exit
Within a Process
exec
SYSCALL_EXECVE
USER_MODE_EXECVE
syscall_exit
Within a Process
syscall_entry(ptregs_execve)
SYSCALL_SCHED_TRY_WAKEUP
sched_try_wakeup
SYSCALL_SCHED_SCHEDULE
sched_schedule
SYSCALL_EXIT_GROUP
Within a Process
syscall_entry(sys_exit_group)
SYSCALL_PROCESS_EXIT
SYSCALL_SEND_SIGNAL
process_exit
send_signal
sched_try_wakeup
SYSCALL_SCHED_TRY_WAKEUP
Execution with exec with execve Exit Cloning
SYSCALL_CLONE
process_fork
Within a Process
syscall_entry(ptregs_clone)
SYSCALL_MIGRATE_TASK
sched_migrate_task
SYSCALL_PROCESS_FORK
SYSCALL_WAKEUP_NEW_TASK
sched_wakeup_new_task
syscall_exit
USER_MODE_FORK
sched_schedule (child process)
SYSCALL_SCHED_SCHEDULE
USER_MODE_SHCEDULED
syscall_exit
32
Other Patterns
File Control Read Link Unlink Get Resource Limit New UName
33
Thank You!
Questions?
top related