Parallelizing Security Checks on Commodity Hardware

Ed Nightingale Dan Peek, Peter Chen

Jason Flinn

Microsoft Research University of Michigan

ASPLOS 2008 2

Run-time security checks

Provide run-time intrusion detection/prevention Many powerful checks have been proposed Examples: Taint-analysis, on-access virus

scanning, system call graph modeling

Often impose high performance penalties

What makes run-time checks slow?

ASPLOS 2008 3

Problem

AABCD

Application

1) Begin application2) Pause execution3) Run security check

Application& security

check

4) Resume application

PAUSE

RESUME BPAUSE

RESUME CPAUSE

RESUME DPAUSE

Checks may slow performance by 10x

ASPLOS 2008 4

Solution

Speck (Speculative parallel check) Parallelize checks across many cores Provide security equivalent to sequential check

Use three techniques Transparent kernel replay system Operating system support for speculative

execution Buffer dependent output until checks complete

ASPLOS 2008 5

Accelerating checks with Speck

AB

cpu0 cpu1

cpu2

C

cpu3

D

cpu4

Speculate!

2) Execute check in parallel

1) Run process speculatively

3) Start later checks in parallel to earlier ones

Speed of parallelization without sacrificing safety!

ASPLOS 2008 6

Outline

Introduction

Implementing Speck

Parallelizing security checks

ASPLOS 2008 7

Example: Executing sequentially

All state ‘frozen’ and available to security check Examples – heap, address space, file name,

system call, value of a single address in memory

APAUSE

ASPLOS 2008 8

Example: Executing in parallel

How does security check get app state?

Acpu0 cpu1

fork()

Speck uses fork() to copy state to other core

ASPLOS 2008 9

fork()

Example: Executing in parallel

BCD

cpu0

Acpu1

Bepoch 0

Amortize fork by grouping checks into epochs

What happens when app interacts with OS?

epoch 1fork()

cpu2

E

E

CF

F

ASPLOS 2008 10

Safely executing system calls

Some system calls affect state of calling process Introduces non-determinism in execution

Other system calls may affect other processes Other processes may be compromised as well

Some system calls generate output Output to screen or network cannot be undone

ASPLOS 2008 11

Replaying non-determinism

Speck replays system calls that affect process state Some system calls (mmap) re-executed Signal delivery limited to exit from system call

Prevents instrumented clone from diverging

Uninst Process Inst Clone

Execute read()

datadata

OS

Execute read()

ASPLOS 2008 12

Tracking causal dependencies

Speculator [sosp05] tracks causal dependencies Supports FIFOs, pipes, UNIX sockets etc. Undo log associated with each object Do not handle multi-threading (MP replay

hard) All dependent objects rolled back on failure

Uninstrumented process terminated on failure

Uninstrumented process can safely run ahead

ASPLOS 2008 13

Handling output commits

Problem: Some system calls create output

Output to network or screen buffered

When all checks within epoch complete dependent output released

ASPLOS 2008 14

Equivalence/Safety

Assume attacker cannot compromise Speck in-kernel replay system Speculator in-kernel causal dependency

tracking

Replay system ensures code equivalence Instrumented clone does not diverge

Speculator prevents permanent damage Speculator rolls back all dependent state on

failure

ASPLOS 2008 15

Outline

Introduction

Implementing Speck

Parallelizing security checks

ASPLOS 2008 16

Choosing security checks

All checks depend upon uninstrumented process Amount of state determines Speck strategy Some checks require little state or run

infrequently No need for fork and replay…just pause app and ship

state

Some checks depend upon result of earlier check Independent checks easy to parallelize Many dependencies make parallelization harder Example: taint analysis

ASPLOS 2008 17

Using Speck to parallelize checks Process memory analysis

Sensitive data leaks

System call analysis On-access virus scanner

Data flow analysis Taint analysis

ASPLOS 2008 18

Process Memory Analysis

Checking for transient leaks of sensitive data

Our check examines every memory store Looks for signature of sensitive data Examines all 16 byte windows around

address Use Pin dynamic binary rewriting tool

Later checks do not depend on earlier checks

ASPLOS 2008 19

MPlayer video decoding

Speck 7.5x faster with 8 cores Video plays in real time

1 2 3 4 5 6 7 80

5

10

15

20

25

30

35

40

Number of cores

Speck Sequential

Fra

mes

per

seco

nd

ASPLOS 2008 20

On-access virus scanner

Scan files on-access by a process Many different policies -- on-read on-write etc.

Our check scans each file on-close Emulates news/email server Implemented using ClamAV libraries (180K

sigs)

Little state is required and checks independent Speck does not use fork and replay for this

check

ASPLOS 2008 21

PostMark benchmark

2.8x more TPS

1 2 3 4 5 6 7 80

1000

2000

3000

4000

5000

6000

7000

Number of cores

Speck SequentialTra

nsa

ctio

ns

per

seco

nd

ASPLOS 2008 22

Taint analysis

Trace flow of data from untrusted sources Instrument application at instruction

granularity Update map of tainted addresses at run-time Ensure certain addresses are not tainted

Later checks depend on result of earlier checks Running in parallel cannot tell whether

address was tainted during prior check

ASPLOS 2008 23

Parallelizing taint analysis

New algorithm minimizes sequential processing

fork()

BCD

cpu0

Acpu1A

Bfork()

cpu2

C

D

…cpu n

log

log

ASPLOS 2008 24

Dynamic taint compression

Dependency logs were very large Time to process longer than to run sequential

check

Only care about dependencies before check Addresses and registers often overwritten Eliminate unimportant dependencies Reduce log size 6x

ASPLOS 2008 25

Taint analysis: MPlayer

2x speedup on 8-cores

1 2 3 4 5 6 7 80123456789

10

Number of cores

Speck Sequential Speck no opt

Fra

mes

per

seco

nd

ASPLOS 2008 26

Choosing a strategy

Check State required/When required

Checks independent?

Fork & replay?

Work to make ||?

Memory analysis

MemoryOn store

Yes Yes None

Virus check

File nameOn file close

Yes No A little

Taint analysis

Rep. of addr spaceOn data flow inst.

No Yes A lot

Fork & replay – need to attach check at fork

No fork & replay – need to accept concurrent requests

ASPLOS 2008 27

Conclusion Run-time security checks can be slow

Speck accelerates run-time checks Parallelizing across many cores Provides safety of executing checks

sequentially