basics of windows os for reverse engineering malware...• windows 7 refactored the system libraries...

Protecting the irreplaceable | f-secure.com

Basics of Windows OS for Reverse Engineering Malware

Copyright F-Secure 2010. All rights reserved.

Applications on Windows

February 18, 2013 2 Copyright F-Secure 2010. All rights reserved.

Executable Format

• Object files and executables follow the PE

(Portable Executable) file format

• Full specification available online

• http://www.microsoft.com/whdc/system/platform/

firmware/PECOFF.mspx

• Best viewed with your hex editor (HT) or

specialized PE viewer (PEBrowsePro ->)


http://www.microsoft.com/whdc/system/platform/firmware/PECOFF.mspx

http://www.microsoft.com/whdc/system/platform/firmware/PECOFF.mspx

Windows Executables

• Filename extension hints to the executable type

• EXE = executable application, anything from a DOS executable to 64-bit

Windows applications

• DLL = Dynamic link library, a set of callable routines compiled together as

a loadable file

• SYS = Driver

• OBJ = Object file, input to the linker

• Note that Windows does not really care much about the file extension

• You can execute application.jpg just fine

• All of these follow the PE/COFF specification

• APPX is used for Windows 8 Windows Store apps

• Not a PE, but a ZIP archive with all application contents inside


Windows API

• Windows API (aka. Win32 API) is the interface to the operating system for applications

• Exposed by a set of system libraries: kernel32.dll, user32.dll, …

• Windows 7 refactored the system libraries (“MinWin”) so you will see e.g. kernelbase.dll

• On 64-bit, because of 32-bit compatibilty, c:\windows\syswow64 contains another set of libraries

• Several subcategories

• Administration and management (WMI, …)

• Diagnostics (event logging, …)

• Networking

• Security

• System services (processes, threads, registry…)

• MSDN is the reverse engineers best friend for Windows binaries

• http://msdn2.microsoft.com/en-us/library/default.aspx


Native API

• Undocumented interface to OS functionality

• One level below Windows API

• Some low-level functionality only available through Native API

• Examples of interesting functions

• NtSetSystemInformation

• NtQuerySystemInformation

• NtQueryDirectoryFile

• See “Windows NT/2000 Native API Reference”

by Nebbett


Windows Architecture


Simplified Windows Architecture

February 18, 2013 8

System Support

Processes

Service

Processes

User

Applications

Environment

Subsystems

Subsystem DLL’s

NTDLL.DLL

Executive

Kernel Device Drivers

HAL

Windowing &

Graphics


System Mechanisms


Kernel-mode & user-mode

February 18, 2013 10

User-space

(Ring 3)

System-space

(Ring 0)

User-space

(Ring 3)

User-space

(Ring 3)

0x00000000

0xFFFFFFFF

Int 0x2e /

Sysenter


System Service Dispatching


ReadFile(…)

Call NtReadFile

Return to caller

Sysenter

Return to caller

Call NtReadFile

Dismiss interrupt

Read the file

Return to caller

Interrupt

Application

Kernel32.dll!

ReadFile

Ntdll.dll!

NtReadFile

Nt!

KiSystemService

Nt!

NtReadFile


Processes & Threads


Processes

• Abstraction of an executing program

• A process has

• A private virtual address space (user-mode address space)

• A base image (the main executable)

• A set of open handles to OS resources

• An access token (“who is this process, what can it do”)

• A process ID (PID)

• One or more threads

• Job = A group of processes that can be managed as a unit


Threads

• A thread is what gets scheduled for execution on the CPU

• A thread has

• A context (the state of execution)

• A list of exception handlers

• Two stacks, one for user-mode and one for kernel-mode

• A thread ID (TID)

• An access token

• Thread-Local Storage (TLS), a per-thread storage area

• Fiber = Manually scheduled unit of execution, shares the context of its

owning thread (ConvertThreadToFiber, CreateFiber, SwitchToFiber)


Processes & Threads


Demo: Processes and Threads


Case Study: FU Rootkit

• The FU rootkit hides processes by unlinking them from the kernels process

list

• Why do those processes still get execution time?

© F-Secure Confidential February 18, 2013 19 Copyright F-Secure 2010. All rights reserved.

Case Study: DLL Injection

• A technique to run code in the context of another process by forcing it to load a DLL

• Motivation

1. Stealth: no extra processes in process list

2. Data capture using API hooks (e.g. banking trojans)

3. Making reverse engineering more difficult

4. Avoiding HIPS features (outbound network connectivity from trusted process)

• Various ways to accomplish DLL injection on Windows

• Registry: AppInit_DLLs

• Using remote threads

• SetWindowsHookEx

• From the kernel using APC’s


What does this code do?


TEB & PEB

• TEB = Thread Environment Block

• Container for thread-specific things like the exception handler list, stack pointer, …

• Windows uses the fs segment to store it (offset 0x18 has pointer to self)

• mov eax, fs[18]

• PEB = Process Environment Block

• Container for process-specific things like the list of loaded modules

• TEB has a pointer to PEB at offset 0x30

• Important when reversing code that

• Enumerates loaded modules (Peb.Ldr)

• Checks for an attached debugger (PEB.BeingDebugged)

• Installs an exception handler (TEB.NtTib.ExceptionList)


Example: Checking For a Debugger

; Call IsDebuggerPresent()

call [IsDebuggerPresent]

test eax, eax

; Do the same by checking PEB

mov eax, large fs:18h ; Offset 18h has self-pointer to TEB

mov eax, [eax+30h] ; Offset 30h has pointer to PEB

movzx eax, byte ptr [eax+2] ; PEB.BeingDebugged

test eax, eax


Example: Installing an Exception Handler

; Install a SEH exception handler

push offset_my_handler ; pointer to our handler

push fs:[0] ; pointer to old exception record

mov fs:[0], esp ; update TEB.NtTib.ExceptionList


Memory Management


Memory Manager

• Each process on Windows sees a large, continuous address space

• The memory manager has two important tasks

1. Mapping access to virtual memory into physical memory

2. Paging contents of virtual memory to disk as physical memory runs out,

and paging data back to memory when it’s needed


Virtual Memory

• Each process has private, virtual address space

• Paging = the process of transferring memory contents to and from disk

• Amount of virtual memory can exceed physical memory


Virtual Memory (x86)

• Total of 4 GB virtual memory

• By default, only lower 2 GB

accessible from user-mode

• Upper 2 GB reserved for

kernel-mode and shared

between all processes


Management Mechanisms


Registry

• A directory for storing settings and configuration data for the Operating

System and applications

• Think of it as a huge .ini file

• Basic concepts

• Hive

• Key

• Value

• Hives are just files, most under

%SystemRoot%\System32\Config


Registry Hive Format


Registry Root Keys

• HKEY_LOCAL_MACHINE (HKLM)

• System-wide information

• HKEY_USERS (HKU)

• User-specific settings for all accounts

• HKEY_CURRENT_USER (HKCU)

• A link to the current user under HKEY_USERS

• HKEY_CLASSES_ROOT (HKCR)

• File associations and COM registration, links to HKLM\Software\Classes

• HKEY_PERFORMANCE_DATA

• HKEY_CURRENT_CONFIG

• Current HW profile, links to HKLM\System\CurrentControlSet\Hardware Profiles\Current


Registry and Malware

• Malware wants to survive a reboot

• Registry is the most common way of starting after reboot

• Hundreds of “launchpoints”

• HKLM\Software\Microsoft\Windows\CurrentVersion\Run:MyApp

• HKLM\Software\Microsoft\Windows NT\CurrentVersion\Image File Execution

Options\explorer.exe:Debugger

• Malware also wants to change (security) settings for other components

• Windows Firewall, IE extensions and settings, Windows File Protection, …

• The registry is also a great source for forensic data, for example:

• HKEY_CURRENT_USER\Software\Microsoft\Windows\ShellNoRoam\MUICache

• HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\

UserAssist


Services

• Services are background processes that usually perform a specific task

and require no user-interaction

• For example, Automatic Updates

• Controlled by the Service Control Manager (SCM), services.exe

• Configuration data under HKLM\System\CurrentControlSet\Services

• Different types of services

• Kernel drivers

• Separate process

• Shared process (hosted by svchost.exe)


Demo: Looking at Services


File Systems


Windows File System Formats

• CDFS (cdfs.sys)

• CD-ROM filesystem, considered legacy

• UDF (udfs.sys)

• Universal Disk Format, the standardized format for optical storage

• FAT12, FAT16, FAT32 (fastfat.sys)

• Legacy filesystem, mostly replaced by NTFS

• exFAT (aka. FAT64)

• Adds functionality on top of FAT (file size limit increase, ACL’s)

• NTFS


NTFS

• Native file system for Windows

• Support for advanced features

• ACL’s on files and directories

• Alternate Data Streams

• Disk quotas

• Sparse files

• Compression and encryption

• Soft and hard links

• Transactional semantics


Demo: Alternate Data Streams


Security Mechanisms


Security & Objects

• Almost everything on Windows is an object

• Process, thread, desktop, mutex, …

• Basic concepts

• Security Identifier (SID) is a unique ID for any actor

• “S-1-5-21-525843606-2469437151-111719316-1006” = DOMAIN\user123

• A token identifies the security context of a process

• “Member of Administrators group, can shut down OS”

• Security Descriptor specifies who can do what to an object

• Owner

• Discretionary Access Control List (DACL)

• Privileges


Access Check


Applications in the (Basic) Windows Security Model


win

wo

rd.e

xe

ntdll.dll

kernel32.dll

User

calc

.exe

ntdll.dll

kernel32.dll

User

serv

ice

s.e

xe

ntdll.dll

kernel32.dll

System

Windows kernel

Filesystem

Registry

Sandboxing Applications

• Especially in later Windows versions (Vista, Windows 7), extensions to the security model can be used to isolate less trustworthy applications

• Prevent exploited applications from changing the system

• In practice, requires support from the application itself

• Protected Mode Internet Explorer, Adobe Reader X, Google Chrome

• A process is isolated from the rest of the system with additional mechanisms, e.g.

• Restricted tokens

• See CreateRestrictedToken() on MSDN

• Job objects

• CreateJobObject(), AssignToJobObject(), SetInformationJobObject()

• Integrity Levels

• Security descriptors and tokens are assigned an Integrity Level

• A process with a lower IL cannot modify (sometimes not read) higher IL objects


Windows 8 and Windows Store Apps


basics of windows os for reverse engineering malware...• windows 7 refactored the system libraries...

Documents