COEN 252 Computer Forensics

Investigating Hacker Tools

Program Analysis Given an executable, how do we

find out what it does? Try to find the program online.

Analyze source code to find clues. Search for the name of the program.

Perform source code review. Execute the program in a sandbox.

Some programs can break out of a sandbox / jail.

Program Compilation Compiler

Translates HLL code to Assembly / ILL Assembler

Translates Assembly code to machine language

Linker Creates object code out of several modules. A program usually makes library calls (stdio)

Program Compilation Statically Linked: All library code is part

of the object code Dynamically Linked: Program calls

library functions. (DLL) Stripping: Removes all human-readable

symbols from object code. Combats reverse engineering.

Packing with UPX, etc. upx.sourceforge.net Compresses source code (achieves ratios of

20% - 40%)

Program Compilation

Static compilation needs more memory

Program Analysis Static Analysis:

Determine the type of executable. ELF file in Unix Exe-type in Windows

Symbol Extraction: Use a program like strings to find

symbols left in object code. Names give hints on program. Will not work for stripped files.

Static Program Analysis

Example for strings output:

Program Analysis

Find the program online: Use the name of the file to find online

versions. Use strings to check whether this is a

similar file. Use same compiler to compile the

online version and check for similarity.

Static Program Analysis

Investigate source code Use Reversing Tools:

Disassembler: Decodes binary machine code into a readable

assembly language text IDA-Pro ILDasm (Microsoft .Net IL disassembler)

Static Program Analysis Investigate source code

Use Reversing Tools: Debuggers

Kernel-mode: Component that sits alongside the system’s kernel Allows for stopping and observing the entire system.

User-mode: Attach to a process. Take full control of process.

Tools: OllyDbg WinDbg (MS tool) IDA-Pro Numega-SoftIce (no longer available in isolation)

Static Program Analysis Investigate source code

Use Reversing Tools: Decompilers

Attempt to produce a high-level language source-code-like representation from a binary.

Never completely possible because The compiler removes some information, The compiler optimizes the code.

System Monitoring Tools Filemon TCPView RegMon PortMon WinObj Process Explorer

Static Program Analysis

Investigate source code Executable-Dumping

Dumpbin (MS) PEView PEBrowse Professional

Program Analysis

Using disassembly:

Program Analysis

Static Program Analysis

Artifacts to look for: Names of functions

Especially API functions. Data strings

Names of constant strings Names of directories Identification of compiler

Program Analysis

Static Program Analysis Compilers generate different types of

code for the same HLL feature Function Calls:

Order in which parameters are pushed on stack. Use of certain registers to pass variables. Use of stack / registers to return a value. Division of labor between callee and caller.

This allows us to recognize the compiler with which an executable was created.

Programmers using assembly will not follow the same standards throughout the code.

Hence, we can recognize assembly writers as well.

Dynamic Program Analysis Run the program and see what it is doing. Requires security mechanisms:

Dedicated machine. Not connected to the internet. Or: Virtual machine.

However: Code can recognize whether it is running in VMWare.

E.g. by the internal MAC addresses, …

Transport malware on a non-writable CD / DVD

Dynamic Program Analysis Strace, systrace:

Run the programming, but keep track of the system calls that it makes with parameters.

More relevant calls (Unix): open read write Unlink lstat socket close

Strace has an option that intercepts all network related calls.

Dynamic Program Analysis

Use fport, netstat, … to determine ports opened by the program.

On Windows systems. Use regmon Use ListDlls Use psList

to find out processes created by program.

Dynamic Program Analysis

Intercept communication of program. Need to generate a fake network. E.g.: Static analysis reveals that the

program tries to contact www.evil.org on the IRC port.

Hence, name an additional machine on separated net www.evil.org.

Dynamic Program Analysis

Run program on a debugger. IDA-Pro OllyDbg SoftIce

Dynamic Program Analysis

Do a web-search for unique names.

Program Analysis Malware writers can use antireversing

techniques. Eliminate symbolic information. Encrypt code. Code obfuscation.

Make HLL constructs difficult to understand. Antidebugger Methods:

Use the IsDebuggerPresent API to protect against user-level debuggers.

Use the NTQuerySystemInformation API to determine if a kernel debugger is attached to the system.

Set a trap flag and check whether it is still there. A debugger would “swallow” it.

Put in bogus bytes over which the code jumps. Does not work for all disassemblers.