June 6th, 2010 1

Techno Securityand Digital Investigations

ConferenceJune 6-9, 2010

Myrtle Beach, SC

Demystifying the Microsoft Extended File Demystifying the Microsoft Extended File System (exFAT)System (exFAT)

Robert Shullich CPP, CISSP, CISM, CISA, CGEIT, GSEC, GCFA

June 6th, 2010 2

Agenda

Why a new file system Forensics Relevance Features Advantages Timelines Support Limits Internals

June 6th, 2010 3

Why do we need a new file system?

Current Limits Exhausted Larger volumes (>2TB) Larger files sizes (>4GB) Faster I/O

(UHS-1: 104 MB/2 - UHS-2: 300MB/s) Removable Media Flexibility Extensibility NTFS Features without the overhead

June 6th, 2010 4

Relevance to Forensics Study

Digital Evidence Extraction Finding the evidence Including the hiding places Validation

Daubert Expert Testimony Need to know and understand file org

New Media (SD Cards) will drive exFAT adoption, and the potential for CP investigations.

June 6th, 2010 5

What happens when you have exFAT formatted media and no exFAT support?

June 6th, 2010 6

Forensics Challenges

Linux OS Support Tuxera drivers may help

Mac OS Support Open Source Tools Commercial Tools

Encase FTK

Documentation

June 6th, 2010 7

Disclaimer

The released specification and implementation is Release 1.00 of exFAT

The specification mentions additional features that were not implemented yet, but may at a future time/ Some of these are Windows CE holdovers

Both may be presented today Some directory entries will be skipped

June 6th, 2010 8

International System of Units (SI) Table

File System in powers of 2

Device characteristics in power of 10

Shorthand Longhand Nth Bytes

KiB Kibibyte 210 1024

MiB Mebibyte 220 1024 KiB

GiB Gibibyte 230 1024 MiB

TiB Tebibyte 240 1024 GiB

PiB Pebibyte 250 1024 TiB

EiB Exbibyte 260 1024 PiB

ZiB Zebibyte 270 1024 EiB

YiB Yobibyte 280 1024 ZiB

June 6th, 2010 9

Features of exFAT 1.00

Sector sizes from 512 to 4096 bytes Clusters sizes to 32MiB Subdirectories to 256MiB Built for speed, less overhead than NTFS but

has some of the NTFS features UTC Timestamp Support

Vista/Server 2008 SP2+, XP with KB

Features of exFAT 1.00 (cont’d)

OEM Parameters Sector for device dependent parameters

12 sector VBR, support of larger boot program

Potential capacity to 64ZiB Current support ≈ 128 PiB

Up to 2,796,202 files per subdirectory File Names max to 255 Characters Unicode File Names and Volume Labels

June 6th, 2010 11

Future Features of exFAT

TexFAT (To be released later) Exists in Windows CE Transaction Safe exFAT

ACL (To be released later) Exists in Windows CE

Encryption Support? Not announced, but mentioned how easy to

add

June 6th, 2010 12

MBR Partition Limitations

Microsoft File Systems are limited when stored in a MBR partition

A partition is defined by a Master Boot Record

A MBR uses a 4 byte value for number of sectors

To get the maximum volume size, exFAT cannot be created within a partition

June 6th, 2010 13

Advantages of exFAT

Handle growing capacities in media, increasing capacity to >32 GB.

> 1000 files in a single directory. Speeds up storage allocation processes. Breaks file size 4 GB barrier. Supports interoperability with future desktop

OSs. Provides an extensible format.

June 6th, 2010 14

Key Dates for exFAT September 2006 – Windows CE 6.0 March 2008 – Windows Vista Service Pack 1 January 2009 – Announcement at CES of SDXC specification January 2009 – Windows XP Drivers Available May 2009 – Windows Vista Service Pack 2 August 2009 – Tuxera Signs File System IP Agreement with

Microsoft March 2009 – Pretec Releases first SDXC Cards December 2009 – Microsoft (re)announces exFAT license

program for third-parties December 2009 – SDXC laptops due soon December 2009 – Diskinternals releases exFAT recovery utility December 2009 – Encase support

June 6th, 2010 15

More Key Dates for exFAT

December 2009 Sony, Canon & Sanyo License

January 2010 Funai License (LCD TV) February 2010 Panasonic License February 2010 Panasonic 64/48GB SDXC February 2010 Sony Memory Stick XC February 2010 Sandisk Ultra XC 64GB Card

3.0 Spec $350

More Key Dates

June 1st 2010 Tuxera Releases Linux & Android exFAT drivers

June 3rd 2010 Kingston Releases Class 10 SDXC 64GB Card 60 MB/s read, 35 MB/s write.

June 6th, 2010 17

SD Card Association

New Memory Card Consumer Appliances Follows SDHC Specification for 2TB

Capacity

June 6th, 2010 18

SDXC Storage Capabilities

From 32GB to 2TB on a card Exclusively exFAT File System 300 MB/s I/O Transfer Storage

4,000 RAW images 100 HD movies or 60 hours of HD recording 17,000 fine-grade photos in a single directory

June 6th, 2010 19

Support for exFAT

Windows XP & Server 2003 KB955704

Vista & Server 2008 SP1 Vista & Server 2008 SP2

(Adds UTC timestamp support) Windows 7

June 6th, 2010 20

Reference Standards

Bits are numbered right to left 76543210

Decimal Offsets Little-Endian numbers Unsigned numbers Sectors vs. Clusters Strings are 16 bit Unicode Strings not Terminated

June 6th, 2010 21

File System Integrity

Version Verified 3 Checksums

VBR UP-Case Table File Set

Critical Directory Entries Other Checks and Balances File System should NOT mount if failures

June 6th, 2010 22

exFAT Limits

Volume size 128PiB MS said 64ZiB MS now says 256TiB

File Size 16 EiB (64 bit number) Bigger than volume size

Subdirectory 256MiB Sector 512-4096 bytes (29-212) Cluster 32MiB (225) No floppy support No FAT32 minimum cluster (65,525) restriction No 8.3 file name support

June 6th, 2010 23

Data Hide Alert!

FAT32 max cluster 32KiB exFAT max cluster 32MiB Potential for massive slack space

June 6th, 2010 24

Volume Space Layout

The Main Boot Region Contains main VBR

The Backup Boot Region Contains backup VBR

The FAT Region Contains FAT Table(s)

The Data Region (Cluster Heap) This is where data resides

June 6th, 2010 25

VBR – Volume Boot Record

Contains 12 sectors 1 sector main boot sector

Jump Code (3 bytes) BPB (BIOS Parameter Block) Boot Strap Code

8 sectors main extended boot sectors 1 sector OEM parms 1 sector reserved 1 sector VBR Checksum

June 6th, 2010 26

Boot Parameter Block (BPB)

OEM Label “EXFAT ” Volume Length (64-bit) [sector] FAT Location & Size [sector] Heap Location & Size [sector, cluster] Volume Serial Number Location of Root Directory [cluster] Volume Flags Sector and Cluster Sizes [2-shift] Percent in use File System Revision (0x0010=1.00)

June 6th, 2010 27

Sectors & Clusters

A 2-Shift is a power of 2 Sector size and sectors per cluster

Each stored in 1 byte Theoretical maximum is 2255

Sector Size Maximum 212

Sectors per cluster is derived Cluster Size Maximum is 225

June 6th, 2010 28

Executable Boot Code

First 3 bytes of Main Boot Sector Jump Code 0xEB7690

Offset 120 size 390 Remainder of boot code

Offset 510 End signature marker 0xAA55 = “55AA”

Offset 512 Unused if defined

June 6th, 2010 29

More Bootable Code

Up to 8 Main Extended Boot Sectors FAT32 had 3 sector VBR with 1 MEBS Entire sector can be used for boot code Last 8 bytes of sector is marker 0xAA550000 = “000055AA”

Larger capacity for boot virus!

June 6th, 2010 30

VBR Checksum Sector

The 12th sector of the VBR Repeating 4 byte checksum Checksum of previous 11 sectors Flags and Percent excluded

These are volatile and change often Boot Sector Virus & Checksum

June 6th, 2010 31

Offset 0 1 2 3 4 5 6 7 8 9 A B C D E F

00000000 C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹00000010 C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹00000020 C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹00000030 C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹00000040 C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹

Lines 00000050 through 01BF repeated

000001C0 C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹000001D0 C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹000001E0 C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹000001F0 C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹

VBR Checksum Sector

June 6th, 2010 32

FAT – File Allocation Table

When it is used, same as legacy FAT Not used when file contiguous Never used for cluster allocation FAT 32 has 32 bit cells, uses 28 bits exFAT has 32 bit cells, uses 32 bits

There is no 64 bit FAT Maximum clusters is 232-11 With TexFAT – 2 FAT Tables (2 Bitmaps) Addressed by pointer in VBR Size stored in VBR

June 6th, 2010 33

Cell Values in FAT Table

0x00000000 – No significant meaning 0x00000001 – Not a valid cell value 0xFFFFFFF6 – Largest Value 0xFFFFFFF7 – Bad Block 0xFFFFFFF8 – Media Descriptor

Fixed Disk 0xFFFFFFF9-0xFFFFFFFE – Not Defined 0xFFFFFFFF – End of File (EOF)

June 6th, 2010 34

FAT Table Example

Offset 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

0000 F8 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 0010 FF FF FF FF 00 00 00 00 00 00 00 00 00 00 00 000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 000040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 000060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 000080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0000A0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0000C0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0000E0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 000100 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Media ReservedUP-Case TableAllocation Bit Map

Root Directory

June 6th, 2010 35

Allocation Bitmap

Keeps track of cluster allocation status Zero – Free Cluster One – Allocated Cluster

1 Byte = Tracking of 8 Clusters Bit Zero – Byte Zero = Cluster 2

Cluster 0 & Cluster 1 are not defined Addressed by Directory Entry With TexFAT – 2 of these (FAT Pairing)

June 6th, 2010 36

Data Hide Alert!

The Allocation Bitmap and the UP-Case Table are stored as files, and provide hiding space in the metadata

These files are static, typically won’t move, and have slack space.

Nothing prevents someone from moving these files elsewhere in the cluster heap, and actually making them larger

June 6th, 2010 37

June 6th, 2010 38

Directories in exFAT

Root (VBR Pointer) Contains certain critical entries Almost unlimited in size

Subdirectory (by File Entry) Contains file sets 256MiB Max size No physical “.” or “..” entries

Uses 16 Bit Unicode for strings Every Entry 32 bytes in size Entry 0x00 is end of directory Has capabilities for user entries

June 6th, 2010 39

Data Hide Alert!

Manipulation of the Allocation Bitmap, and creation of user directory entries provides the capability of hiding a file system within the file system

It may also be possible to hide data within the directory metadata itself

June 6th, 2010 40

Entry Type

Type Field Offset (Bits) Size (Bits)

In Use 7 1

Category 6 1

Importance 5 1

Code 0 5

June 6th, 2010 41

Entry Type

In Use: 0 – Not in Use, 1- In Use

Category: 0 – Primary, 1 – Secondary

Importance: 0 – Critical, 1 – Benign

Code: Identifies the entry

June 6th, 2010 42

Volume Label Directory Entry

0x83 or 0x03 Entry Primary Entry Only resident in Root Directory Contains the Volume Label 16 bit Unicode 0x03 means no volume label

June 6th, 2010 43

Offset 0 1 2 3 4 5 6 7 8 9 A B C D E F

00000000 83 0A 65 00 78 00 46 00 41 00 54 00 2D 00 31 00 ƒ.e.x.F.A.T.-.1.00000010 32 00 38 00 4B 00 00 00 00 00 00 00 00 00 00 00 2.8.K...........

Volume Label Directory Entry

Type

Volume Name Length (10)

Volume Label (exFAT-128K)

June 6th, 2010 44

Allocation Bitmap Directory Entry

0x81 Entry Primary Entry Only resident in Root Directory Points to the Allocation Bitmap

If TexFAT, then 2 of these Flag bits says which FAT/Bitmap

Cluster Address of Bitmap Size of Bitmap

June 6th, 2010 45

Allocation Bitmap Directory Entry

Offset 0 1 2 3 4 5 6 7 8 9 A B C D E F

0000 81 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0010 00 00 00 00 02 00 00 00 3F 00 00 00 00 00 00 00

Type Cluster Address (Cluster 2) Size (63 bytes)

June 6th, 2010 46

UP-Case Table Directory Entry

0x82 Entry Primary Entry Only resident in Root Directory File names are case insensitive Used to fold file name Table has a checksum (32 bits)

June 6th, 2010 47

UP-Case Table Directory Entry

Offset 0 1 2 3 4 5 6 7 8 9 A B C D E F

0000 82 00 00 00 0D D3 19 E6 00 00 00 00 00 00 00 00 0010 00 00 00 00 03 00 00 00 CC 16 00 00 00 00 00 00

Type Cluster Address (3)

Length (0x16CC = 5,836)Table Checksum

June 6th, 2010 48

File Directory Entry Set

Used to define a file May have 3 to 19 entries, or more 1 Primary, many Secondary Is considered an array

Must be in order Must be contiguous (no gaps)

Entire Set has Checksum

June 6th, 2010 49

File Directory Entry

0x85 or 0x05 Entry Primary Entry Set Checksum (16 bits)

Not modified on file delete Secondary Count

# Secondary entries that follow File Attributes Timestamps

June 6th, 2010 50

Timestamps & Time Zones

3 Timestamps (MAC) 32 bit DOS Date/Time

Local Machine Time 10ms Offset (MC) TZ Offset (MAC)

15 minute increments 7 bit signed number ±16 hours Present with UTC support

June 6th, 2010 51

Timestamp Accuracy

FAT32 – Last Access – Date only exFAT – Last Access – Date/Time All DOS DATE/TIME Double Seconds 10ms adds 0-1990 ms to time 10ms only for Create/Modify

June 6th, 2010 52

Timestamp Reliability

Timestamps appear to be updated when the file is created or modified.

Last Accessed Timestamp appear to be updated when file is created or modified.

Last Accessed Timestamp appear NOT modified on file read.

Forensics Implication on MAC time analysis

June 6th, 2010 53

File Attributes

Attribute Offset Size Mask

Reserved2 6 10

Archive 5 1 0x20

Directory 4 1 0x10

Reserved1 3 1

System 2 1 0x04

Hidden 1 1 0x02

Read-Only 0 1 0x01

June 6th, 2010 54

File Directory Entry

Offset 0 1 2 3 4 5 6 7 8 9 A B C D E F

0000 85 04 D4 92 20 00 00 00 44 62 86 3B F1 62 BA 3A 0010 44 62 86 3B A8 00 EC EC EC 00 00 00 00 00 00 00

Type # Secondary Entries

Set Checksum (0x92D4)

Attributes (0x0020 = Archive)

Create

Modified

TZ Offset CMA EC = GMT-5

Accessed

Create 10ms

Modified 10ms

June 6th, 2010 55

Formatted File Directory Entry

Root Entry Type Read is: 85 Directory Entry RecordChecksum: 92D4Calculated Checksum is: 92D4 Size Directory Set (bytes): 160Secondary Count 004File Attributes: 0020 Archive Create Timestamp: 3B866244 12/06/2009 12:18:08Last Modified Timestamp: 3ABA62F1 05/26/2009 12:23:34Last Accessed Timestamp: 3B866244 12/06/2009 12:18:08 10 ms Offset Create A8 168 10 ms Offset Modified 00 0 Time Zone Create EC 236 Value of tz is: GMT -05:00 Time Zone Modified EC 236 Value of tz is: GMT -05:00 Time Zone Last Accessed EC 236 Value of tz is: GMT -05:00

June 6th, 2010 56

Stream Extension Directory Entry

0xC0 or 0x40 Entry Secondary Entry Length of Name Length of File (2 of them) Cluster address of first data block Name Search Hash value Secondary Flag

FAT Invalid Allocation Possible

June 6th, 2010 57

Stream Extension Directory Entry

Offset 0 1 2 3 4 5 6 7 8 9 A B C D E F

0000 C0 03 00 28 AD 3C 00 00 1F 46 1D 01 00 00 00 000010 00 00 00 00 05 00 00 00 1F 46 1D 01 00 00 00 00

Entry Flags (Alloc Possible/Fat Invalid)

Length of File Name (0x28= 40)

Name Hash (0x3CAD)

Cluster (5)

Data Length 0x011d461f = 18,695,711

June 6th, 2010 58

Parameters for Samples

Bytes Per Sector: 2 to the 09 power is: 512Sectors Per Cluster: 2 to the 08 power is: 256Bytes per Cluster: 131072 (128K)

June 6th, 2010 59

Formatted Stream Extension

Root Entry Type Read is: C0 Directory Entry Record, Stream ExtensionSecondary Flags: 03 Flag Bit 0: Allocation Possible Flag Bit 1: FAT Chain InvalidLength of UniCode Filename is: 40Name Hash Value is: AD3CStream Extension First Cluster 5Cluster 5 is AllocatedStream Extension Data Length 18695711 Bytes Slack: 83487 Clusters Used: 143Stream Extension Valid Data Length 18695711 Bytes Slack: 83487 Clusters Used: 143

June 6th, 2010 60

File Name Extension Directory Entry

0xC1 or 0x41 Entry Secondary Entry Secondary Flags

Allocation not possible FAT Invalid

15 Characters (30 bytes) of Name Name in 16 Bit Unicode In order (FAT32 LFN was reversed) Up to 17 max, total 255 character

June 6th, 2010 61

File Name Extension Directory Entry

Offset 0 1 2 3 4 5 6 7 8 9 A B C D E F

0000 C1 00 62 00 75 00 73 00 69 00 6E 00 65 00 73 00 Á.b.u.s.i.n.e.s.0010 73 00 5F 00 6F 00 66 00 5F 00 73 00 65 00 63 00 s._.o.f._.s.e.c.

0000 C1 00 75 00 72 00 69 00 74 00 79 00 5F 00 5F 00 Á.u.r.i.t.y._._.0010 62 00 75 00 73 00 2D 00 31 00 30 00 35 00 2D 00 b.u.s.-.1.0.5.-.

0000 C1 00 33 00 32 00 6B 00 62 00 70 00 73 00 2E 00 Á.3.2.k.b.p.s...0010 6D 00 70 00 33 00 00 00 00 00 00 00 00 00 00 00 m.p.3...........

File Name = business_of_security__bus-105-32kbps.mp3

June 6th, 2010 62

Significance of “not in use” flag

0x05, 0x40 & 0x41 Entries “Not in use” may mean deleted files May also be reallocated rename

Set Checksum not changed when entries marked “not in use”

June 6th, 2010 63

Summary

exFAT is a new generation of the FAT family of Microsoft File Systems

The need for forensics tools will heat up in 2010

We don’t have the right tools yet Documentation and support for exFAT is

scarce

June 6th, 2010 64

Q&A

June 6th, 2010 65

Contact Information

E-mail: rshullic@earthlink.net Blog: rshullic.wordpress.com Blog: shullich.blogspot.com

June 6th, 2010 66

References

Sans Reading Room:

http://www.sans.org/reading_room/whitepapers/forensics/rss/reverse_engineering_the_microsoft_exfat_file_system_33274

Microsoft Patent:

Microsoft Patent 0164440 (June 25, 2009). Quick Filename Lookup Using Name Hash.

Pub No. US 2009/0164440 A1 Retrieved December 10, 2009 from

http://www.pat2pdf.org/patents/pat20090164440.pdf