VMM Based Rootkit Detection on Android

Class PresentationPete Bohman, Adam Kunk, Erik Shaw

Problem Statement

• Rootkit detection and prevention on the Android platform with specific regards to the sensitive resources Android provides.• Kapersky 2011: 1046 unique malware strains

targeting mobile platforms• Android platform built on Linux Kernel, a well known

target.• Sensitive information on smart phones• GPS, contacts, text messages, call log

Defensive Rootkit Approaches• User mode rootkits

– Process infection, binary patching, lib hooks

• User mode integrity checkers– tripwire, chkrootkit, rkthunter, AV scanner

• Kernel mode rootkits– malicious device drivers and lkms– sys call hooking, kernel data structure manipulation

• Kernel level inspection– behavioral analysis, data structure integrity checkers, hook detection

• But… Any kernel level inspection mechanisms can be subverted by kernel level rootkits

Our Approach• Two Pronged• VMM inspection • Ensures integrity of static kernel module• Isolated from host OS

• Kernel module security mechanisms • System call integrity checks• Hidden process detection• Android capability table

• We exercise a “layer-below” level of security in which we establish trust beneath the kernel

Solution PreviewAndroid Software Stack

Android VMM ensures integrity of static kernel module

Kernel module implements security mechanisms

Overview

• Design• VMM Design• Protected KM Design• Implementation• Results• Demo Presentation• Conclusion• Q&A

VMM Interface Design

Android VMM

Hardware (Emulator)

Linux Kernel

Trusted KM <ISR>

Libraries and Runtime

Application Framework

1. Hardware Timer Interrupt

2. Validate Protected KM 3. Raise Monitor Interrupt

4. Invoke KM

Protection KM Design

Linux KernelTrusted KMSystem Call WhitelistOriginal Sys Call Table

Libraries and Runtime

Maps

Application Framework

Contacts SMS Phone

Content Provider

Location Provider

Activity Manager

Open

Malicious Native

ApplicationSystem Calls

Open SocketRead GPSSQL Query

Malicious LKM

SysCall Table

Implementation

• Instead of utilizing a VMM as the lowest layer, we created a new hardware device in the QEMU emulator• Hardware device based on virtualized timer which

expires at predefined intervals• When a timer interval expires, an interrupt is

generated and the timer is reset• During each interrupt, the hardware device calls into

our kernel module to pass execution• Integrity checks are done on the sys_call_table

Implementation (cont.)

• We implemented a static Kernel Module (KM) in order to make integrity checks on the sys_call_table• The kernel module contains a copy of the original

sys_call_table and compares this to the running version

• The KM also sets its own hooks in the sys_call_table in order to detect malicious access to sensitive resources• e.g. Check for invalid process access to sys_open system call

attempting to open a sensitive resource file (contacts2.db)

Results

• We are able to detect/correct modifications to the sys_call_table• We are able to prevent malicious access to sensitive

resources• TODO Mention Malicious App and LKM

• TODO: (Insert link to demo)

Conclusion

• Layer Below Protection• Security of the Linux kernel must be rooted in a layer

below the kernel• Code contained solely in the kernel is subject to any

kernel-level attack

• Sensitive Resource Protection• Android mobile phones contain lots of sensitive

information that must be protected

App PermissionsDalvik VM IsolationApp signatures

Limits application abilities in order to prevent malicious behavior.

Virus ScannersRemote Lockout

Modified system binariesTrojan’d servicesStolen device

Linux user and group permissions

Access control

Architecture Layer Security Mechanism Threat Mitigation

Linux Kernel Source with

Protection KMCompilation

Kernel Image

Protected Text Sect.

Protected Data Sect.

QEMU Emulator(VMM) Compilation

QEMU Emulator(VMM)Protected TextProtected Data