© 2004 microsoft corporation. all rights reserved. 1 filter manager overview

© 2004 Microsoft Corporation. All rights reserved. 1

Filter Manager Overview


Problems with today’s Filter Model Poor control over filter load order

Non-deterministic Increases test matrix Causes interop problems

No unload support Requires reboot

Must process all operations ~5000 lines of code to start


Problems with today’s Filter Model (cont) Kernel stack overflow

Call-through model Filters generate recursive IO

Inefficiencies due to redundant work in filters Each filter does its own name lookup,

normalization, and name caching Implement their own context support Don’t always get it correct


Problems with today’s Filter Model (cont) Complex interfaces Difficult to add new file system

operations (IRP/FastIO) Existing filters must be modified

New IRP and FastIO operations were added to W2K

Broke all existing filters TxF (new Longhorn feature)

Multiple OS version support Filter must be developed to “least common

denominator” API set


The Filter Manager addresses all of these

Issues


What is the Filter Manager

Legacy file system filter Reduces the complexity of IO

system through new interfaces and library routines

Has kernel and user-mode interfaces


Filter Manager

User Mode IO Operations

I/O Manager

FAT NTFS RDR

Filter Manager Frame 1 (1000-9999)

IRP + FastIO + FsFilter Interfaces

Legacy Filter Driver


Minifilter

Minifilter

Minifilter

Minifilter

Minifilter


Definition of Terms

Legacy Filter A file system filter developed to the

current Filter model Minifilter

A file system filter developed to the Filter Manager model


Definition of Terms Instance

A filters attachment to a volume at a particular altitude

Multiple filter instances per volume is supported See MiniSpy

Altitude A unique identifier which defines the relation of this

filter to other filters An “infinite precision” string with a decimal point

Example: 4501.345 Can always insert a new filter between two existing

filters All minifilters must have a unique altitude Altitudes are managed by Microsoft

Developing a web site for requesting altitudes. Altitude determines relative stack position


Instances and Altitudes Instance: A filters

attachment to a volume at a particular altitude

Support multiple instances of a minifilter on a volume

Altitude determines relative stack position

Volumec:

AntiVirus Filter(Altitude: “300”)

Encryption Filter(Altitude: “100”)

Volume

“LanmanRedirector”

AntiVirus Filter(Altitude: “300”)

Encryption Filter(Altitude: “100”)

Conceptual IO Flow

Conceptual IO Flow

MiniSpy Filter(Altitude: “200”)

MiniSpy Filter(Altitude: “400”)


Definition of Terms

Frame Multiple filter manager attachments

to a file system stack to support proper interop with legacy filters

Each frame defines an altitude range


Filter Manager with Multiple Frames


I/O Manager

FAT NTFS RDR





Minifilter

Minifilter

Minifilter

Minifilter

Minifilter


Definition of Terms

CallbackData Filter manager's equivalent of an IRP All operations come to minifilters via

this structure Iopb

Io Parameter Block Points to current stack location


Problems with today’s Filter Model (Review) Poor control over filter load order No unload support Must process all operations Kernel stack overflow Inefficiencies due to redundant work in

filters Complex interfaces Difficult to add new operations Cross version support


How these problems are addressed

Poor control over filter load order Minifilter can be loaded at any time Minifilters can be inserted into the

middle of the attachment chain Altitude determines relative stack

position Deterministic load order


How these problems are addressed (cont)

No unload support Minifilters can be unloaded at any

time Filter Manager synchronizes the safe

removal of all minifilter attachments through notifications

Filter Manager handles operations which complete after the minifilter unloads



Must process all operations Minifilter registers only for operations

in which it is interested Can uniquely register for pre- or post-

Operation callbacks Can ignore certain classes of

Operations Paging IO Cached IO



Kernel stack overflows Filter Manager uses a “callback” model

instead of a “call-through” model Supports non re-entrant filter initiated

i/o IO will only be seen by filters below your

filter Filter Manager is optimized to reduce

the amount of stack it consumes


Filter Manager’s Callback model


I/O Manager

FAT NTFS RDR

Filter Manager Frame 1



Filter Manager Frame 0

Minifilter

Minifilter

Minifilter

Minifilter

Minifilter



Inefficiencies due to redundant work in filters Filter Manager provided infrastructure

Name generation is time consuming Filter Manager caches file names for use

by multiple filters Filter Manager is optimized to support

multi-processor systems


How these problems are addressed (cont) Complex interfaces

Filter Manager simplifies complexity by providing support routines for common functionality

Naming Support Context Support User/Kernel mode communication Masking differences between file systems

Filter Manager eliminates complexity by automatically handling certain functionality

Enumerating and attaching to file system stacks IoMarkIrpPending()



Difficult to add new operations Only filter manager needs to be

updated to support new operations Minifilters only register for those

operations they are interested in


How these problems are addressed (cont) Multiple OS version support

Filter Manager supports running of minifilters on down level OS’s

Can register for operations that don’t exist

Can programmatically determine which APIs exist

FltGetRoutineAddress() Structures are designed to be extensible

Registration structure has a version number Structures contain a size field


Additional Filter Manager Features


Context Management Allows minifilters to attach private

information to a system object Volume Context Instance Context File Context (not implemented yet)

Associated with all opens across all data streams for a given file

Stream Context Associated with all opens for a given data stream

of a file Stream Handle Context

Associated with a single open of a single data stream


Name Management

Copy-free retrieval of file names Retrieves names in one of three

formats: Normalized, opened, short

When possible, name is cached Filter manager manages name

lifetime Handles renames, hardlinks, etc.


Filter Initiated IO

IO is targeted to lower filters Io is not recursive

Properly integrated with minifilter unload


User/Kernel Communication

Minifilter creates a named port Minifilter controls who can access

the port via ACLs Bi-directional communication

channel


fltmc.exe

Command line utility for common filter management operations Load and unload minifilters Attach/detach minifilters to/from volumes Enumerate minifilters, instances, volumes

“fltmc help” Displays help information


Debugging Help fltkd debugger extension

Part of debugger package .load fltkd !fltkd.help

List all available commands For more specific help on a single

command, issue that command with no parameters

Important commands !cbd

Filter Manager equivalent to !irp !frames

List all Frames and their filters


Debugging Help fltkd debugger extension (cont)

!filters List all Frames and their filters

!volumes List all volumes and their instances

!volume, !filter, !instance Give detailed information on the specific object

Run with debug version of fltmgr.sys Lots of ASSERT to catch common errors


Filter Verifier Enable via Driver Verifier (verifier.exe)

Select minifilter driver name Enable “I/O Verification“ option

Recommend verifier value: 0x7b Verification starts when a filter registers with the Filter

Manager Validates all Filter Manager APIs

Validates parameters and calling context Verifies return values from minifilter’s pre/post callback

routines Ensures minifilter changed the parameters in the callback

data in a coherent/consistent manner Lots more to come

Always develop minifilters with driver verifier and filter verifier enabled!


Thoughts about the Filter Manager

It is much simpler to start developing a filter. Eliminates ~5000 lines of

infrastructure code It is still just as hard to complete a

production quality filter due to the complexity of the IO system


Filter Manager Changes/Enhancements


Filter Manager Changes(May 2004) Modified APIs

User/Kernel Communication Removed: FltClosePort() Added: PFLT_PORT for port handles Added: FltCloseClientPort() Added: FltCloseCommuniciationPort()

FltCancelFileOpen() IoCancelFileOpen() bug has been fixed

FltPerformAsynchronousIo() Can now be used on FSCTL operations


Filter Manager Changes(May 2004)

Following fields were modified to be a CONST FLT_CALLBACK_DATA.Thread FLT_CALLBACK_DATA.Iopb FLT_RELATED_OBJECTS.*


New APIs (May 2004) Added lock APIs for EResources

FltAcquireResourceExclusive() FltAcquireResourceShared() FltReleaseResource()

Use existing Ex routines to init and delete Acquire/release wrapped by KeEnter/LeaveCriticalRegion

Added lock APIs for PushLocks FltInitializePushLock() FltDeletePushLock() FltAcquirePushLockExclusive() FltAcquirePushLockShared() FltReleasePushLock()

Acquire/release wrapped by KeEnter/LeaveCriticalRegion


New APIs (May 2004) FsRtl Byte Range Lock package support

Added new routines FltAllocateFileLock() FltCheckLockForReadAccess() FltCheckLockForWriteAccess() FltFreeFileLock() FltInitializeFileLock() FltProcessFileLock() FltUninitializeFileLock()

Use in conjunction with existing routines

Allows minifilter to process byte range lock operations


New APIs (May 2004)

New APIs FltRequestOperationStatusCallback()

Used to request a callback which receives the return value from IoCallDriver()

Necessary if you need to know if an oplock was granted or not

FltIsOperationSynchronous() FltSetSecurityObject()


New APIs (May 2004)

New APIs FltIs32bitProcess()

Always returns TRUE on 32bit processors FltCreateSystemVolumeInformationFolder()

Will create the “System Volume Information” folder if it does not already exist.

Created with correct ACLs


New APIs (Nov 2003) FltReferenceContext()

Add reference to context object Call FltReleaseContext() to remove

reference FltReferenceFileNameInformation()

Add reference to name information object

Call FltReleaseFileNameInformation() to remove reference


New APIs (Nov 2003) FltGetRoutineAddress()

Retrieves address of filter manager routines by name Used for dynamic detection of new filter manager

APIs FltIsIoCanceled()

Returns if the given operation has been canceled or not

FltNotifyFilterChangeDirectory() See FsRtlNotifyFilterChangeDirectory() Allows minifilter to process directory change

notification operations


New APIs (Nov 2003) Additional APIs which support non-

recursive operations: FltQueryInformationFile() FltSetInformationFile() FltQueryVolumeInformationFile() FltQuerySecurityObject() FltFlushBuffers() FltFsControlFile() FltDeviceIoControlFile()

See ZW equivalents


New APIs (Apr 2003) FltGetFileNameInformationUnsafe()

Allows you to retrieve a file name given a file object and an instance

Should only use when you know query is safe from deadlocking system

FltCancelFileOpen() No longer return special status from post-

Create callback Equivalent of IoCancelFileOpen() IoCancelFileOpen() bug has been fixed


New APIs (Apr 2003)

FltIsVolumeWritable() FltEnumerateVolumeInformation() FltGetRequestorProcess() and FltGetRequestorProcessId() Work the same as their Io

equivalents, but take a FLT_CALLBACK_DATA structure


Three New Minifilter Samples(May 2004) MetadataManager

Shows what a minifilter needs to do to manage a private metadata file

Handles volume lock/unlock Handles volume mount/dismount including surprise

removal Ctx

Shows how to use Instance, Stream, and StreamHandle contexts

CDO Shows how to write a minifilter which needs to have a

control device object Shows how to handle filter load/unload requests


Release Plan

Already in XP SP2 Already in Srv03 SP1 Already in Longhorn Running prototype for W2K

Working on final approval for a QFE which runs on W2K SP4

© 2004 microsoft corporation. all rights reserved. 1 filter manager overview

Documents

new filter

filter manager model

time filter manager

notifications filter

encryption filter altitude

filter manager addresses

filter manager overview

microsoft corporation