process out-grafting: an efficient “out-of-vm” …...process out-grafting: an efficient...
TRANSCRIPT
Process Out-Grafting:
An Efficient “Out-of-VM” Approach for
Fine-Grained Process Execution Monitoring
Deepa Srinivasan, Zhi Wang, Xuxian Jiang, Dongyan Xu*North Carolina State University, Purdue University*
Workshop for Frontiers of Cloud Computing, Dec 1, 2011, IBM T.J. Watson Research Center, NY
Malware Infection Trend
New malware samples collected by McAfee Labs, by month*
*Figure source: McAfee Threats Report: Second Quarter 2011, McAfee Labs 2
Anti-Malware Isolation
� Traditional anti-malware tools are not well-isolated
� Virtual Machine (VM) introspection
� Isolate tool by placing it outside a VM
� Analyze states and events externally
User-mode Applications
…
Hypervisor
Virtual
Machine
Monitor
3
OS Kernel
Anti-Malware Isolation
� Traditional anti-malware tools are not well-isolated
� Virtual Machine (VM) introspection
� Isolate tool by placing it outside a VM
� Analyze states and events externally
User-mode Applications
…
Hypervisor
Virtual
Machine
VM
Introspection
Monitor
4
OS Kernel
Out-of-VM Solutions
� Livewire (Garfinkel et al., NDSS ‘03)
� XenAccess (Payne et al., ACSAC ‘07)
� VMScope (Jiang et al., RAID ‘07)
� Lares (Payne et al., Oakland ‘08)
� …
5
Semantic Gap in Introspection
� What we want to observe
� High-level states and events (e.g. system calls, processes)
� What we can observe
� Low-level states and events (e.g. raw memory, interrupts)
OS Kernel
User-mode Applications
…
Hypervisor
Virtual
Machine
Internal
Monitor
External
Monitor
Semantic
Gap
6
Addressing the Semantic Gap
� Guest view casting
� VMWatcher (Jiang et al., CCS ‘07)
� Automatic generation of introspection-based tools
� Virtuoso (Dolan-Gavitt et al., Oakland ‘11)
� Issues
� Sensitive to internal OS changes or updates
� Incompatible with existing anti-malware tools
7
Our Goal
Support existing in-host process monitors
out-of-VM without semantic gap!
8
In-host strace
9
In-host Monitors
� Process-level monitoring
� System calls
� Library calls
� Instruction execution traces
10
Process Out-Grafting
� Isolation
� Monitor protection from malware
� Compatibility
� Natural support for fine-grained user-mode process
monitoring tools (strace, ltrace, …)
� Efficiency
� No significant performance overhead due to isolation
11
Rest of This Talk
� Motivation
� System Design
� Implementation & Evaluation
� Related Work
� Conclusion
12
Assumptions
� Trusted hypervisor
� Untrusted monitored VM
� Non-goals
� OS kernel-level monitoring
� Stealthy monitoring
13
Key Techniques
Hypervisor
Production VM Security VM
User
Kernel
User
Kernel
Suspect ProcessStub
Technique I: On-Demand Grafting
Technique II: Mode-sensitive Split Execution
Monitor
14
Key Techniques
Hypervisor
Production VM Security VM
User
Kernel
User
Kernel
Technique I: On-Demand Grafting
Monitor
15
On-Demand Grafting
On-Demand Grafting
� Relocate suspect process to security VM
� Enable efficient, native inspection
� Eliminate hypervisor intervention
� Support existing process monitoring tools
� Initiate out-grafting as needed
� Restore process after monitoring
17
When to Out-Graft?
18
� Process in user-mode
� Process in kernel-mode
� Hypervisor notified when user-mode execution resumes
Hypervisor
Production
VM
User
Kernel
NXPhysical
Memory
Frames
Suspect Process
What to Out-Graft?
19
Hypervisor
Production
VM
User
KernelEIP
EAX
EBX
…
…
� Architecture-specific resources
� No OS kernel-specific resources
� Main root cause in semantic gap
� Continued execution of out-grafted process
Network
SocketDisk File
Physical
Memory
Frames
Suspect Process
How to Out-Graft?
Hypervisor
Production VM Security VM
User
Kernel
User
Kernel
Suspect ProcessStub
NX X
Helper Module
Monitor
VCPU Register State
Page Mappings
Stub protected
by hypervisor
20
Physical
Memory
Frames
Mode-Sensitive Split Execution
Mode-Sensitive Split Execution
� All user-mode execution occurs in security VM
� All kernel-mode execution occurs in production VM
� Out-grafted process considers itself running in production
VM
22
System Call Redirection
� Smooth, continued execution of out-grafted process
� Monitor isolation
Hypervisor (e.g., KVM)
Production VM Security VM
User
Kernel
User
Kernel
Suspect ProcessStub
Helper Module
Monitor
Disk File
Network
Socket
System
Call
- System call
number
- Arguments
No hypervisor
intervention!
23
Page Fault Forwarding
� Consistent process memory mapping between VMs
� No semantic knowledge in security VM (e.g. memory-mapped
file)
Hypervisor (e.g., KVM)
Production VM Security VM
User
Kernel
User
Kernel
Suspect ProcessStub
Helper Module
Monitor
- Faulting
address
- Read/Write
fault
Page
Fault
Page Tables
Page
Fault
24
Page Fault Forwarding
� Consistent process memory mapping between VMs
� No semantic knowledge in security VM (e.g. memory-mapped
file)
Hypervisor (e.g., KVM)
Production VM Security VM
User
Kernel
User
Kernel
Suspect ProcessStub
Helper Module
Monitor
Page
Fault
Page Tables
Page
Fault
25
Physical
Memory
Frame
Rest of This Talk
� Motivation
� System Design
� Implementation & Evaluation
� Related Work
� Conclusion
26
Implementation
� Hypervisor: KVM (2.6.36.1)
� Out-grafting support: +1309 SLOC
� Extended page tables support (Intel Core i7)
� Host OS: Ubuntu 10.04 (kernel 2.6.28)
� Guest OS: Ubuntu 9.04, Fedora 10
27
Security Analysis
Hypervisor (e.g., KVM)
Production VM Security VM
User
Kernel
User
Kernel
Suspect ProcessStub Monitor
Disk File
Network
Socket System
Call
28
� Monitor isolation and effectiveness
� System call forwarding
� Security VM protected from kernel attacks
Security Analysis
Hypervisor (e.g., KVM)
Production VM Security VM
User
Kernel
User
Kernel
Suspect ProcessStub Monitor
29
� Stub protection
� Hypervisor-protected memory and page tables
Security Analysis
30
� Out-grafting detection
� Strong policy for random, arbitrary out-grafting
Case Studies
� thttpd as test process for out-grafting
� Both disk and network usage
� Support existing in-host tools, out-of-VM
� strace, ltrace, gdb
� OmniUnpack (Martignoni et al., ACSAC ‘07)
31
Case Studies
� thttpd as test process for out-grafting
� Both disk and network usage
� Support existing in-host tools, out-of-VM
� strace, ltrace, gdb
� OmniUnpack (Martignoni et al., ACSAC ‘07)
32
strace
33
In-VM strace Out-of-VM strace
Code Unpacking Detection
� OmniUnpack (Martignoni et al., ACSAC ‘07)
� Track page writes and executions
� Detect unpacking when executing a previously-written
page
� Faithful reproduction of algorithm in Linux
� Kernel module in security VM
34
Localized overhead in
out-of-VM monitoring
Thorough test of page
fault forwarding
NX bit support only in
security VM kernel
Performance
� Dell T1500, Intel Core i7, 4 cores, 2.6 GHz, 4 GB
RAM
� VM configuration
� Production VM: 1 VCPU, 2047 MB RAM
� Security VM: 1 VCPU, 1 GB RAM
35
Performance
36
� Inter-VM system call (getpid): ~11 µs
� Process state identification: ~250 µs
� Slowdown to out-grafted process
� File-copy time: 35.42%
� thttpd throughput: 7.38%
Performance
Production VM Slowdown with a Contending Process Out-grafted
-6%
-4%
-2%
0%
2%
4%
6%
8%
37
Rest of This Talk
� Motivation
� System Design
� Implementation & Evaluation
� Related Work
� Conclusion
38
Related Work
� VM introspection
� Livewire (Garfinkel et al., NDSS ‘03), XenAccess (Payne et al., ACSAC ‘07), VMScope (Jiang et al., RAID ‘07), Lares (Payne et al., Oakland ‘08), VMWatcher (Jiang et al., CCS ‘07), Virtuoso (Dolan-Gavitt et al., Oakland ‘11)
� Efficient, isolated monitoring
� SIM (Sharif et al., CCS ’09)
� Process migration
� BLCR (Smith, UCB-TR ‘08), Zap (Osman et al., OSDI ‘02), Migration survey (Nuttall et al., ACM OS Review ’94)
� System call forwarding
� Application protection (Ta-Min et al., OSDI ‘06), Monitoring fidelity (Martignoni et al., ICISS ‘09)
� Sandboxing, isolation techniques
� Ostia (Garfinkel et al., NDSS ‘04), Janus (Goldberg et al., Security ‘96)
39
Conclusion
Hypervisor
Production VM Security VM
User
Kernel
User
Kernel
Suspect ProcessStub
Technique I: On-Demand Grafting
Technique II: Mode-sensitive Split Execution
Monitor
40
Process Out-grafting: An Efficient Out-of-VM Approach for
Fine-grained Process Execution Monitoring (CCS ‘11)
Thank you!
Questions?