armin bahramshahry vancouver, b.c. v7p 3g4 april …armin bahramshahry 1501-2024 fullerton ave....

Armin Bahramshahry

1501-2024 Fullerton Ave.

Vancouver, B.C. V7P 3G4

April 16, 2007

Dr. Matei Ripeanu

Electrical and Computer Engineering

The University of British Columbia

2332 Main Mall

Vancouver, BC V6T 1Z4

Dear Dr. Ripeanu,

Please find attached my final report for the EECE 496 course project. The report contains

a description of the context, in which my work was performed, and includes an

explanation of my research, design and implementation for this project. I also have

mentioned a set of recommendations for future work.

I look forward to your feedback.

Thanks,

Armin Bahramshahry.

c.c. J. Pavelich

UBC Electrical & Computer Engineering Department

vanDisk: An Exploration in Peer-To-Peer Collaborative Back-up Storage SystemEECE 496 Term Paper Submitted to: Ms. Jane Pavelich and Dr. Matei Ripeanu

Armin Bahramshahry 4/16/2007

ii

ABSTRACT

This term paper discusses the research, initial setup, and development of a new user interface

component for the currently existing reliable peer-to-peer backup storage system developed by

EECE 496 students.

The objective of the project is to study the existing user-interface of the vanDisk project for

compatibility with another project named “free-loader”. There has been research done on other

technologies provided by Microsoft for Windows operating systems to design and implement a

virtual drive with file system callbacks.

The requirement of such component is to provide common “file-level” file system callback

interface for both the vanDisk project and the free-loader project. It needs to have a reasonable

throughput and CPU consumption. It is also very important to keep in mind the response time of

the system.

This report explains the research on four technologies offered by Microsoft named, Kernel-Mode

Driver Framework (KMDF), User-Mode Driver Framework (UMDF), Windows Shell

Namespace, and Installable File System (IFS). Later it explains the design decisions that have

been made, and it will explain the initial design and development. Following which it will

demonstrate the experiments that have performed on the developed functionalities and their

results. Finally it will make a set of recommendations on the findings and conclude on how this

project could be continued and how the findings are useful.

iii

Table of Contents

ABSTRACT .....................................................................................................................................ii LIST OF FIGURES ......................................................................................................................... v LIST OF TABLES .......................................................................................................................... vi GLOSSARY .................................................................................................................................. vii LIST OF ACRONYMS ................................................................................................................ viii 1.0 INTRODUCTION ............................................................................................................... 1

2.0 BACKGROUND ................................................................................................................. 3 3.0 RESEARCH ......................................................................................................................... 4

3.1 Windows Kernel-Mode Driver Framework ..................................................................... 4 3.1.1 Framework Overview ............................................................................................... 4 3.1.2 I/O Flow Model......................................................................................................... 5 3.1.3 Build Process ............................................................................................................ 7 3.1.4 Installation Process ................................................................................................... 8 3.1.5 Debug Process ........................................................................................................... 8

3.2 Windows User-Mode Driver Framework ........................................................................ 9 3.2.1 Framework Overview ............................................................................................... 9 3.2.2 I/O Flow Model....................................................................................................... 10 3.2.3 UMDF Objects ........................................................................................................ 11 3.2.4 UMDF Callback Interfaces ..................................................................................... 13

3.3 Windows Shell Namespace ............................................................................................ 14 3.4 Windows Installable File System ................................................................................... 15

3.4.1 Overview ................................................................................................................. 15 3.4.2 Filter Manager Benefits .......................................................................................... 16

3.4.3 Architecture ............................................................................................................. 17 3.4.4 I/O Flow Model....................................................................................................... 18

4.0 DESIGN AND IMPLEMENTATION .............................................................................. 20 4.1 Design............................................................................................................................. 20

4.1.1 Overview ................................................................................................................. 20 4.1.2 Minispy .................................................................................................................. 20 4.1.2 Initial Design ........................................................................................................... 21

4.2 Implementation............................................................................................................... 22

iv

5.0 EXPERIMENTS ................................................................................................................ 24 5.1 Experiments .................................................................................................................... 24 5.2 Results and Analysis ...................................................................................................... 24

6.0 RECOMMENDATION ..................................................................................................... 28

7.0 CONCLUSION .................................................................................................................. 29 8.0 APPENDIX A .................................................................................................................... 30 9.0 APPENDIX B .................................................................................................................... 32 10.0 APPENDIX C .................................................................................................................... 34 11.0 APPENDIX D .................................................................................................................... 40 12.0 REFERENCES .................................................................................................................. 42

v

LIST OF FIGURES

Figure 1 Lifecycle failure pattern for hard drives [2]. .................................................................... 1 Figure 2 KMDF I/O Flow [3] ......................................................................................................... 6 Figure 3 I/O Flow to UMDF Driver [6] ........................................................................................ 11 Figure 4 Simplified I/O Stack with Filter Manager and Minifilter Drivers [9] ............................ 18 Figure 5 File System Design ......................................................................................................... 22 Figure 6 Message Polling Stress Test Graph ................................................................................ 25 Figure 7 Message Polling Stress Test Graph at 0 Intervals .......................................................... 26 Figure 8 Message Polling CPU Usage Graph ............................................................................... 27

vi

LIST OF TABLES Table 1 Supporting Create, Cleanup, and Close Requests ............................................................ 10 Table 2 UMDF Object Types [6] .................................................................................................. 12 Table 3 Driver Callback Objects and Interfaces [6] ..................................................................... 14 Table 4 KM-UM Message Passing Stress Test Result ................................................................. 31 Table 5 KM-UM Message Passing CPU Consumption ................................................................ 33 Table 6 Mini-Filter Major Functions [10] .................................................................................... 39 Table 7 Batch File Description ..................................................................................................... 41

vii

GLOSSARY

Term or Phrase Definition

Block Level In data storage, a block is a group of records on a storage device.

Blocks are manipulated as units. For example, disk drives often

read and write data in 512- byte blocks.

Callback In computer science, a callback is executable code that is passed as

a parameter to other code. It allows a low level software layer to

call a function occurring in a higher level layer.

Device Driver A software module that manages access to a hardware device,

taking the device in and out of service, setting hardware

parameters, transmitting data between memory and the device,

sometimes scheduling multiple uses of the device on behalf of

multiple processes, and handling I/O errors.

Filter Driver A file system filter driver intercepts requests targeted at a file

system or another file system filter driver. By intercepting the

request before it reaches its intended target, the filter driver can

extend or replace functionality provided by the original target of the

request.

Kernel The fundamental part of a program, typically an operating system,

that resides in memory at all times and provides the basic services.

It is the part of the operating system that is closest to the machine

and may activate the hardware directly or interface to another

software layer that drives the hardware.

Polling A technique that continually interrogates a peripheral device to see

if it has data to transfer.

viii

LIST OF ACRONYMS

CPU ............................................................................................................. Central Processing Unit

DMA ............................................................................................................ Direct Memory Access

DDI ............................................................................................................ Device Driver Interfaces

FSFilter ................................................................................................................. File System Filter

GUID....................................................................................................... Globally Unique Identifier

IFS ................................................................................................................. Installable File System

INF .......................................................................................................................... Information File

IRP ...................................................................................................................... I/O Request Packet

KM ............................................................................................................................... Kernel Mode

KMDF ........................................................................................... Kernel-Mode Driver Framework

OS ......................................................................................................................... Operating System

P2P ................................................................................................................................. Peer-to-Peer

PC ........................................................................................................................ Personal Computer

UM ................................................................................................................................... User Mode

UMDF .............................................................................................. User-Mode Driver Framework

WDM .......................................................................................................... Windows Driver Model

WMI ................................................................................... Windows Management Instrumentation

1

1.0 INTRODUCTION

Today computers are used more often than ever in society, corporations, and government

organizations. Other than many security risks involving loss or manipulation of personal and

important data, there are many others less obvious but more disastrous factors that are not

considered by most individuals and organizations. One of the factors that are much related to the

topic of this project is the reliability of the storage systems. Hard drives tend to fail and they

actually have a well known “useful-life period” pattern shown in figure 1 and considering them

as infinitely reliable can be very costly.

Figure 1 Lifecycle failure pattern for hard drives [2].

The vanDisk project provides a storage system with a much higher reliability, meaning lower

risks for the users of its system. VanDisk also provides higher performance compare to local

hard drive since clients are able to perform read and write operations in parallel. VanDisk

provides these functionalities by replicating data over multiple servers, and reconstructing the

data and re-replicating it upon any failures. The core components of this system are implemented

and are operating; however, there are some improvements and new features that have to be

implemented. The user interface of this system is currently providing information at “block

level” which doesn’t provide enough information for better management on the client side than

the existing implementation; it is also not compatible with the free-loader project. This project is

focused on providing such interface at “file system level”, which provides enough information

for all sorts of management on the client side and it is also satisfying the needs of the free-loader

project.

2

This report presents an investigation of a windows operating system (OS) specific user interface

for the vanDisk project, considering that this component would also be integrated to another

ongoing project called “Free-Loader” [1]. A major part of this investigation was to research and

study different technologies that provide functionality to server these purposes. This report

contains a summary of my exploration of such systems.

My objectives during this project are first to attempt in changing the existing user interface to

provide the new functionalities required, part of which is to check for compatibility with the free-

loader project. In case that the existing code does not satisfy the project’s requirements, further

study on other technologies that would enable implementation of such interface has to be done.

The next step after having a solid decision on which technology to use, is to design and

implement initial functionalities based on the selected technology.

Through my investigation of the existing system I realized that it could not be modified for use

by free-loader since free loader requires information at file system level. Afterwards I studied

four technologies offered by Microsoft to implement such functionalities. Later I have found the

best fit solution for the specified requirements and began the initial design and implementation of

this system. Later I measured the performance of the system that I implemented by running few

stress tests, all of which are explained in detail through this report.

This report divides into the following primary sections. The section 2 provides background

information and system overview of the vanDisk project. The section 3 is focused on my

research on the four technologies offered by Microsoft for such problems. The section 4

explains the initial design choices and issues of implementing the chosen technology. Section 5

is focused on implementation and experiments, I make few recommendations in section 6 and

section 7 concludes my findings.

3

2.0 BACKGROUND

VanDisk provides different set of configurations, every user of the system can be the client or the

owner of the data to be stored, and it also can be a peer-node which provides storage space for

other clients. One type of configuration is where only selected trusted nodes that are

administrated by an organization are the peer-nodes. Another type of configuration would be

where every node is both the client and peer-node forming a completely distributed backup

storage system.

The system provides a very reliable storage space for its clients by replicating the client’s data

over multiple peer-nodes. It is also providing a very friendly and easy to use interface by

mounting a new logical volume that operates just like other local hard disk volumes. The

difference is that vanDisk logical volume operates over the network with a number of selected

peer-nodes to perform read and write operations. These peer nodes are responsible for securely

and consistently respond to client’s request.

The vanDisk operations are currently performed at “block level” meaning there are no

information regarding the type of the operation nor there any information on the directory

structures or file information. The vanDisk system encrypts the data before writing it to a peer-

node and accordingly decrypts the block upon read operations; these operations are handled by

TrueCrypt which also implements the user interface for vanDisk.

There is a new feature under development to secure the integrity of the data by appending the

hash code of the blocks at its end enabling the client to check for modifications by peer-node.

There is also another ongoing project on providing caching mechanism on the client side. To

have full control over all operations, and to improve the caching mechanism as well as the

security level, we require information at file level and not block level. We also like to remove

and later replace the encryption and decryption algorithms by the TrueCrypt with our own

implemented security system. To satisfy all these needs while considering the free-loader project

in mind, we need to implement a new user interface intercepting file system calls at file level.

4

3.0 RESEARCH

The research is focused on analyzing different technologies that provides functionalities to

mount a logical drive to the user’s work space, which also provides functionality to intercept the

file system calls at file level. To do so first I studied the implementation of the TrueCrypt, and I

realized that they develop a logical driver that is implemented using the Kernel-Mode Driver

Framework (KMDF). Based on the information I have gathered I began my research by looking

at the KMDF technology which later led me to look at the User-Mode Driver Framework

(UMDF). Following my research on these two technologies I have acted upon Dr. Ripeanu’s

suggestion to study the Windows Shell Namespace and the Windows Installable File System

(IFS) as well. Bellow is a detail explanation of these four different technologies, their

architecture and functionality.

3.1 Windows Kernel-Mode Driver Framework

The Kernel-Mode Driver Framework (KMDF) is a driver framework developed by the Microsoft

to make the process of creating and maintaining Kernel mode device drivers easier. KMDF is

one of the frameworks included in the Windows Driver Foundation (WDF). It is worth

mentioning that KMDF was originally released to only support Windows XP and Windows

Server 2003 [3], and later in KMDF version 1.1 Microsoft have added the support for Windows

2000 as well. The current version of the KMDF is 1.5.

3.1.1 Framework Overview

The benefit of using KMDF is that all the functionalities of kernel mode programming is

provided, while developers can take advantage of many provided predefined functionalities by

the KMDF. Some of the fundamental features that are provided by the KMDF are the followings:

• Plug and Play and power management

5

• I/O queues

• Direct memory access (DMA)

• Windows management instrumentation (WMI)

• Synchronization

In addition, KMDF offers many management functionalities. For example, KMDF provides a

well-defined object model and controls the lifetime of the objects as well as their memory

allocation and de-allocation.

Developers of Kernel device drivers need to develop a minimal set of operations since KMDF

provides functionality for the default common operations; through which Microsoft can ensure

that the drivers are compatible with each successive release of Windows operating system. In

this design the functionalities that required to be developed are device specific event responses.

3.1.2 I/O Flow Model

Understanding the I/O request and response model is one of the very main steps in writing a file

system driver. Following figure shows the overall flow of the I/O through the KMDF library into

and out of the device driver implemented on top of the KMDF.

6

Figure 2 KMDF I/O Flow [3]

In the first step the IRP is directed to one of the following components by KMDF for processing:

• I/O request handler, responsible for handling request involving device I/O

• Plug and Play/power request handler, responsible for requests involving change in the

device status

• WMI handler, responsible for event-tracing requests or system control requests

Based on the type of the request each of the components mentioned above take one of the

following actions:

• Raise one or more events to the driver

• Forward the request to an internal handler or a target I/O handler for further processing

• Complete the request based on its own action

• Complete the request based on the drivers completion of the request

If any request is not processed when it reaches the end of the framework processing, KMDF

either takes an action and responds instead of the driver according to the driver’s type, or returns

7

the STATUS_INVALID_DEVICE_REQUEST status. In general KMDF automatically forwards

the requests to the default I/O target for “filter drivers”.

3.1.3 Build Process

Even though KMDF supports new Device Driver Interfaces (DDI) and programming models, it

still has a lot in common with WDM [4]. Based on this fact there are some limitations on

development of KMDF device drivers. Following is a short list of few main points:

• Drivers are normally written in C. C++ can be used for driver development in only a very

limited way. Usage of some basic C++ features like inline variable declaration would be safe,

but the object-oriented features of C++ may produce generated code that is not guaranteed to

work correctly in kernel mode [4].

• The C++ compiler works fine with C code and provides better error detection and type safety

than the C compiler [4].

• Drivers must be built with the WDK or DDK build tools. Microsoft Visual Studio is not

designed to support driver development and can be used in only a limited way [4].

KMDF provides two types of build tools, one is the checked build tool and the other one is the

free build tool. Checked builds are similar to the debug builds that are used in application

development. They generate detailed debugging information and enable certain types of

debugging-related code such as ASSERT macros. Checked builds are normally used during the

earlier stages of driver development because they are much easier to debug than free builds.

Keep in mind that due to the support for debugging, checked builds are slower compare to the

free builds.

Free builds are similar to the Release builds that are used in application development. They do

not support detailed debugging information; however, they are fully optimized. Free builds are

8

more difficult to debug, and thus they are used at the end of development cycle where the final

testing and performance tuning is happening.

3.1.4 Installation Process

Drivers need to be installed before they can be used for debugging or end user purposes.

Installing the Kernel Device Drivers that are developed on top of KMDF differ greatly from the

installation of drivers which use installation applications. To install a KMDF driver package, it

needs to contain at least the following three files:

• The KDMF co-installer dynamic-link library (DLL)

• The INF file

• The digitally signed catalog (.cat) file (not required for testing and debugging)

Driver packages also can optionally have files such as icons, property sheet providers, supporting

DLLS, and more.

3.1.5 Debug Process

One of the most effective factors in the speed and efficiency of a development process is the

correct usage of available debugger tools. All programs under development have bugs at some

point, especially in the early stages. Debuggers are also very helpful in a sense that they can be

used as a learning tool, since the developers are able to step through the code and understand

different structures and behaviors of the system.

9

Mainly the debuggers are used to determine where and why the driver is failing, and the only

exception to this rule is the kernel debuggers which can be used to analyze crash-dump files.

Debugging kernel code is a very complicated process, and it requires three hardware

components. One is the host computer running the “WinDbg” [5] debugger. This is usually the

computer used to develop and build the kernel mode device driver. The second component is the

test computer running an appropriate build of Windows that also has the driver installed and the

kernel debugging feature enabled. The third component is a communication system between the

host and the test computer. There are three options for this component:

• Serial ports on the host and test computers with a null-modem cable

• USB

• IEEE 1394 cables

In conclusion, the setup of such environment requires two set of computer or support for virtual

machines that is costly. Also even after the setup, this process is not easy to use and not as

friendly as other debugging tools.

3.2 Windows User-Mode Driver Framework

3.2.1 Framework Overview

The User-Mode Driver Framework (UMDF) is first introduced for development of certain

classes of device drivers for Microsoft’s Windows Vista operating system, and later it was also

made available for Windows XP. The problem with development of completely Kernel Mode

device driver is that a badly written driver can cause major damages to a system since it has very

high privileges and direct access to the kernel. However, development of a driver on top of

UMDF requires the developers to use a dedicated API instead of direct access to the kernel. This

10

is important since upon any critical error the system can restart the driver and prevent it from

damaging other components of the system.

UMDF is a COM based DLL but it doesn’t use the COM runtime library and it only uses the

COM as a programming pattern. UMDF doesn’t even use COM for loading, unloading or for

controlling concurrency [7].

UMDF drivers typically have to implement create, cleanup and close operations. In cases where

these functionalities are not implemented by the driver, UMDF automatically forwards the

request to the default I/O target (lower driver in the device stack) [7]. To support create, cleanup

and close requests, drivers have to implement the following interfaces:

Request type Interface Method

Create IQueueCallbackCreate OnCreateFile

Cleanup IFileCallbackCleanup OnCleanupFile

Close IFileCallbackClose OnCloseFile Table 1 Supporting Create, Cleanup, and Close Requests


UMDF I/O flow is relatively simpler compare to KMDF I/O flow. Following figure shows the

path that an I/O request takes from an application to a UMDF driver. Callers use the same system

calls and Win32 API to send I/O requests to the UMDF driver as they use to send to any other

device. Based on the I/O routine, the Windows kernel I/O manager creates an I/O request packer

(IRP) and sends this IRP to the driver at the top of the kernel-mode device stack for the target

device.

11

Figure 3 I/O Flow to UMDF Driver [6]

It is worth mentioning that UMDF manages the I/O cancellation and completion, and ensures

that the Plug and Play and power state of the device is compatible with performing device I/O.

Based on the type of the I/O request, the UMDF either queues the request or invokes a callback

provided by the driver. UMDF provides configurable I/O queue objects, and driver has to specify

which types of requests to place in each queue and how they should be dispatched. The driver

has to provide callback interfaces to handle I/O requests on its queues.

3.2.3 UMDF Objects

UMDF manages a series of objects that are exposed to the user-mode driver. UMDF creates

some of these objects in response to application-triggered actions, such as an I/O request; the

driver creates other objects by calling methods on UMDF interfaces.

12

For each type of object, UMDF defines one or more interfaces through which to manipulate

instances of the object. The interfaces provide methods and properties. Methods define actions

that can be taken on behalf of the object and return a status to indicate whether they succeeded or

failed. Property operations set and get the attributes of the object and cannot fail. Some interfaces

are implemented by UMDF, and others are implemented by the driver. Following table list all

different objects supported by UMDF and their interfaces and description.

Type of object Interfaces Description

Base object IWDFObject Exposes a base object for use as the driver requires.

Device IWDFDevice Exposes an instance of a device object. A driver typically has one device object for each device that it controls.

Driver IWDFDriver Exposes the driver object itself. Every driver has one driver object.

File IWDFFile

Exposes a framework file object that was opened by the Win32 CreateFile function, through which applications can access the device.

IWDFDriverCreatedFile Exposes a framework file object that the driver created.

I/O queue IWDFIoQueue Exposes an I/O queue, which controls the flow of I/O in the driver. A driver can have any number of I/O queues.

I/O request IWDFIoRequest Exposes a request for device I/O.

I/O target IWDFIoTarget Represents the next-lower driver in the device stack, to which the driver sends I/O requests.

Memory IWDFMemory Exposes memory that the driver uses, typically an input or output buffer that is associated with an I/O request.

USB device IWDFUsbTargetDevice Exposes a USB device object that is an I/O target. Inherits from IWdfIoTarget.

USB interface IWDFUsbInterface Exposes an interface on a USB device.

USB pipe IWDFUsbTargetPipe Exposes a USB pipe that is an I/O target. Inherits from IWdfIoTarget. Table 2 UMDF Object Types [6]

13

3.2.4 UMDF Callback Interfaces

UMDF drivers have to implement callback interfaces to provide specific responses to requests.

Each interface is implemented on a specific UMDF object type. Bellow there is a table of

callback interfaces and the type of object they associate with their brief descriptions.

Type of object Callback Interfaces Description

Base object, or any object that inherits from

the base object type

IObjectCleanup

Provides processing that is required before an object is deleted, typically releasing any references held on the object.

Driver IDriverEntry Provides main entry point and methods to initialize the driver and add its devices.

Device

IPnpCallback Handles device stop, removal, and power state changes.

IPnpCallbackHardware Provides hardware-related operations before device power-up and after device power-down.

IPnpCallbackSelfManagedIo

Provides driver, rather than framework, control over I/O operations at specific Plug and Play and power management states.

IFileCallbackCleanup Handles clean-up requests for file objects on a specific device.

IFileCallbackClose Handles close requests for file objects on a specific device.

I/O queue

IQueueCallbackCreate Handles file create requests.

IQueueCallbackDefaultIoHandler

Handles create, device I/O control, read, and write requests for which no other interface has been implemented.

14

IQueueCallbackDeviceIoControl Handles device I/O control requests.

IQueueCallbackIoResume Resumes processing an I/O request after its queue has been stopped.

IQueueCallbackIoStop Stops processing an I/O request because its queue is stopping.

IQueueCallbackRead Handles read requests.

IQueueCallbackWrite Handles write requests.

I/O request

IImpersonateCallback Provides impersonation for certain Win32 I/O operations.

IRequestCallbackCancel Handles cancellation of an I/O request.

IRequestCallbackRequestCompletion Handles completion of an I/O request. Table 3 Driver Callback Objects and Interfaces [6]

Drivers only need to implement callbacks for the interfaces that are important to their operations.

Upon a request if the corresponding callback is introduced it will be invoked by UMDF,

otherwise the default callback for that request will be invoked.

3.3 Windows Shell Namespace

The Windows Shell namespace is the system of organization that Windows Explorer uses to

present the file system and other objects to the user. It enables the developers to implement

virtual objects like “My Network Places” and “Recycle Bin” and represent them just like real file

system and folders. One of the main responsibilities of the Shell is managing and providing

access to many objects that make up the system. The most common set of these objects are the

folders and files that exist on computer disk drives.

The other set of objects are the virtual objects that do not involve physical storage at all. The

printer object, for instance, contains a collection of links to networked printers. Other virtual

15

objects, such as the Recycle Bin, may contain data that is stored on a disk drive, but needs to be

handled differently than normal files. Virtual objects may even be located on remote computers.

Similar to the file system the two primary types of objects for the shell namespace are files and

folders. Folders are split into two categories, real folders and virtual folders, and like file system

folders, the collection of virtual folders generally varies from system to system. On the other

hand unlike file system folders, users are not able to create new virtual folders, and they can only

be installed. In general there are three classes of virtual folders:

• Standard virtual folders, such as the Recycle Bin, that are found on all systems.

• Optional virtual folders that have standard names and functionality, but may not be

present on all systems.

• Non-standard folders that are installed by the user.

Windows Shell namespace introduces three main interfaces named IPersistFolder, IShellFolder

and IDataObject. The IPersistFolder and IShellFolder provide enough functionality to create a

virtual folder on the user’s system, and the IDataObject provides lots of functionalities two of

which are “SetData” and “GetData” that is enough for our purposes. Windows shell namespace

provides less information compare to UMDF and KMDF, and it lacks a major required property.

It only works on top of Windows Explorer. This prevents applications and command line scripts

to perform operation on a set of files or folders through the implemented virtual folders. To show

such limitation, I have implemented a virtual folder using the IShellFolder interface, and I have

tried to perform operations from command line and failed to do so. This is a major drawback for

our system and in conclusion it is not possible to implement the vanDisk user interface using

windows shell namespace technology.

3.4 Windows Installable File System

3.4.1 Overview

16

The Installable File System (IFS) is a file system API in Microsoft Windows NT and successors

that enables the operating system to recognize and load drivers for file systems. IFS kit has

always supported the development and installation of file system filter driver (legacy filter). A

file system filter driver intercepts requests targeted at a file system or another file system filter

driver. By intercepting the request before it reaches its intended target, the filter driver can

extend or replace functionality provided by the original target of the request. Examples of file

system filter drivers include anti-virus filters, backup agents, and encryption products.

Recently Microsoft has introduced a new feature to its IFS kit named “Filter Manager”. The

Filter Manager is a file system filter driver provided by Microsoft that simplifies the

development of third-party filter drivers and solves many of the problems with the existing

legacy filter driver model.

3.4.2 Filter Manager Benefits

A filter driver developed based on the Filter Manager model is called a minifilter. There are

many benefits in developing a file system driver using minifilter model instead of legacy filter

concept, some of which are listed bellow:

• Better control over filter loads and unloads order: Unlike a legacy filter driver, a

minifilter driver can be loaded or unloaded at any time, while legacy filters can only be

loaded at startup.

• Ability to process only necessary operations: The filter manager uses a callback model

in which a minifilter driver can choose which types of I/O operations to filter. Appendix

C contains the list of major operation that can be chosen by a minifilter.

• Reduced complexity: The filter manager simplifies filtering I/O requests by providing

support routines for common functionality, such as naming, context management,

communication between user mode and kernel mode, and masking differences between

file systems [9].

17

• Easier addition of new operations: Because minifilter drivers register only for the I/O

operations they will handle, support for new operations can be added to the filter manager

without breaking existing minifilter drivers.

• Better support for multiple platforms: A minifilter driver can run on any version of

Windows that supports the filter manager. If a minifilter driver registers for an I/O

operation that isn't available at runtime, the filter manager simply doesn't call the

minifilter driver for that operation. A minifilter driver can determine programmatically

whether functions are available, and filter manager structures are designed to be

extensible [9].

• Better support for user-mode applications: The filter manager provides common

functionality for user-mode services and control programs that work with minifilter

drivers. The filter manager user-mode library, Filterlib.dll, enables communication

between a user-mode service or control program and a minifilter driver. Filterlib.dll also

provides interfaces for management tools.

3.4.3 Architecture

The filter manager is installed with Windows, but it becomes active only when a minifilter driver

is loaded. The filter manager attaches to the file system stack for a target volume. A minifilter

driver attaches to the file system stack indirectly, by registering with the filter manager for the

I/O operations the minifilter driver chooses to filter.

A legacy filter driver's position in the file system I/O stack relative to other filter drivers is

determined at system startup by its load order group. For example, an antivirus filter driver

should be higher in the stack than a replication filter driver, so it can detect viruses and disinfect

files before they are replicated to remote servers. Therefore, filter drivers in the file system filter

(FSFilter) Anti-Virus load order group are loaded before filter drivers in the FSFilter Replication

group. Each load order group has a corresponding system-defined class and class GUID used in

the INF file for the filter driver [9].

18

Like legacy filter drivers, minifilter drivers attach in a particular order. However, the order of

attachment is determined by a unique identifier called an altitude. The attachment of a minifilter

driver at a particular altitude on a particular volume is called an instance of the minifilter driver.

A minifilter driver's altitude ensures that the instance of the minifilter driver is always loaded at

the appropriate location relative to other minifilter driver instances, and it determines the order in

which the filter manager calls the minifilter driver to handle I/O. Altitudes are allocated and

managed by Microsoft [9].

The following figure shows a simplified I/O stack with the filter manager and three minifilter

drivers:

Figure 4 Simplified I/O Stack with Filter Manager and Minifilter Drivers [9]


A minifilter driver can filter IRP-based I/O operations as well as fast I/O and file system filter

callback operations. For each of the I/O operations it chooses to filter, a minifilter driver can

19

register a pre-operation callback routine, a post-operation callback routine, or both [9]. When

handling an I/O operation, the filter manager calls the appropriate callback routine for each

minifilter driver that registered for that operation. When that callback routine returns, the filter

manager calls the appropriate callback routine for the next minifilter driver that registered for the

operation.

For example, assuming all three minifilter drivers in the above figure registered for the same I/O

operation, the filter manager would call their pre-operation callback routines in order of altitude

from highest to lowest (A, B, C), then forward the I/O request to the next-lower driver for further

processing. When the filter manager receives the I/O request for completion, it calls each

minifilter driver's post-operation callback routines in reverse order, from lowest to highest (C, B,

A).

For interoperability with legacy filter drivers, the filter manager can attach filter device objects to

a file system I/O stack in more than one location. Each of the filter manager's filter device

objects is called a frame. From the perspective of a legacy filter driver, each filter manager frame

is just another legacy filter driver.

20

4.0 DESIGN AND IMPLEMENTATION

4.1 Design

4.1.1 Overview

The major part of the design of this new user interface is to choose the technology that would

properly support development of this component. After a complete research on these four

different technologies I have concluded that Windows Shell Namespace does not satisfy the

needs of this project. I also have concluded that both the KMDF and UMDF technologies

support enough functionality to implement our required functionalities. In terms of using KMDF

as it has been discussed above, it is not as safe as UMDF since any small bug could cause critical

damages to the system, and it also very difficult to debug. UMDF is the better choice between

the three technologies mentioned; however, after studying the features of the IFS kit it is

completely clear that IFS is the better choice of all.

IFS contains the benefits of both UMDF and KMDF while providing a safer environment

compare to KMDF and much more functionality compare to UMDF. It also provides a reliable

and very fast communication system between kernel mode (KM) application and the user mode

(UM) application, which can be very useful. Based on these facts the design of the vanDisk user

interface is based on IFS technology.

In order to have a better understanding of how minifilters are designed and implemented I have

studied one of the samples provided by the IFS kit named “minispy”.

4.1.2 Minispy

Minispy is a minifilter provided as an example within the IFS package. It is implemented to

simply spy on a user specified logical drive (can be a virtual drive). This example shows how a

21

minifilter loads and unloads as well as a demonstration on how it can attach itself to a drive and

later detach. This is really important since within this model the two process of virtual drive

creation and the FSFilter development are completely separated and independent.

Minispy registers itself for the complete set of operations for the attached drive and provides

functionality to log the operations to screen or file based on user’s desire. Minispy is consist of

two separate processes, one is the FSFilter driver which is implemented in KM. The other one is

the UM application which is only responsible for interaction with the user. Minispy is a really

useful example to study since it also creates a communication layer between the KM application

and the UM application. Using this communication layer user commands such as list, attach, and

detach are sent to the KM and the logging messages are passed to the UM application from KM

driver.

4.1.2 Initial Design

Using IFS kit the process of development for virtual drive creation is completely separated from

implementation of the vanDisk’s file system. This report doesn’t focus on how the virtual drive

could be implemented and instead it is mostly focused on the design and implementation of the

vanDisk’s file system.

The vanDisk’s file system can follow the same design as the minispy example. This is a

relatively safe design in terms of development as well, because it is minimizing the KM

development. In this design the FSFilter driver developed in KM only intercepts the file system

calls and passes them to the UM application. The UM application can share a common interface

with both the vanDisk project as well as the free-loader project, and each of these projects can

simply extend the shared interface to service their needs.

Based on the proposed design, the user interface is consisting of a separate project which creates

a virtual drive, and another project implementing the file system. The main components of this

file system is the FSFilter driver, the UM file system, the shared interface and a fast

22

communication layer between the FSFilter driver and the UM file system. Following diagram

shows the initial design:

Figure 5 File System Design

4.2 Implementation

Taking advantage of the minispy code which is a very simple FSFilter driver just observing the

file system calls and possibly logging them based on user’s request, the next important

component of the design represented in figure 5 is the communication layer. The communication

layer requires to be able to pass the IRP from KM filter driver to the UM file system application.

It also requires a well defined structure in order to consistently pass all messages back and forth.

Initially I tried to simply modify the existing minispy code to pass specific messages from KM

driver to UM application. In doing so I have encountered an “unknown bug”, I would like to

emphasize on the fact that this bug until today is unknown for the fact that I did not have a

proper debugging environment setup, and due to this bug Windows crashed after running the

program for few seconds. I do believe that one reason that this attempt has failed is that I tried to

modify their fairly complicated communication system for the vanDisk purposes, which most

probably have caused an unexpected situation.

23

Faced with two options to continue the implementation, I had to choose between setting up the

debugging environment and attempting to implement the communication layer from scratch

without any debugging tools. I decided to implement this layer even though it was risky due to

the fact that with facing any bug my attempt would have been a failure without debugging tools.

I managed to successfully implement this layer for a simple message passing initially. This

system is based on a polling algorithm by the UM application. The KM driver or the “server”

registers to a specific port name, to which the UM application connects. Following that the UM

application can send messages to the KM driver through a buffer passed as “input buffer”. In

cases where the UM application is expecting messages from KM application in return it has to

provide a buffer for such messages as well.

Extending the initially very simple implementation of the communication layer, I have

implemented a fixed size queue in the KM driver to be able to keep track of messages for the

UM application until the next poll time.

24

5.0 EXPERIMENTS

5.1 Experiments

There are always few main concerns with using polling algorithms and those are the throughput

of the system, its CPU consumption and the average response time of the system. In order to

measure these values I performed a set of stress tests on the communication layer implemented. I

measured the throughput of the system by passing different buffer sizes to the KM application,

and varying the polling interval.

I gathered information on throughput and CPU consumption of the system for polling intervals

varying from 0 milliseconds to 1000 milliseconds. I performed this test for buffer sizes varying

from 256 bytes (B) to 4 mega byte (MB). I have included a selective set of the result in appendix

A and appendix B. Results in appendix A demonstrates the throughput of the system for passing

as many messages from KM to UM in one second. Results in appendix B are related to the CPU

consumption of the system in percentage for different test cases.

5.2 Results and Analysis

The interesting findings in the throughput results is how the slope of the graph changes among

different buffer sizes focusing on the polling interval of ‘0’ to ‘1’ millisecond. The other

important finding is that there is almost no change in the throughput of the system between the

‘1’ to ‘10’ millisecond polling intervals, I believe this is due to the UM application waiting in the

ready queue of the operating system for CPU time. Figure 6 demonstrates the throughput of the

system in terms of number of 128B messages per second.

25

Figure 6 Message Polling Stress Test Graph

The other very interesting result is that the maximum throughput of the system at ‘0’ polling

interval is reached with 64MB, and passing larger buffer sizes with ‘0’ polling interval only

reduces the throughput. Following figure is over the same set of data as figure 5 focused on the

‘0’ to ‘1’ millisecond polling interval.

1

10

100

1000

10000

100000

1000000

10000000

0.1 1 10 100 1000

# of M

essages Pe

r Second

Polling Interval (ms)

KM‐UM Message Polling Stress Test

256 Buffer Size

512 Buffer Size

1024 Buffer Size

2048 Buffer Size

4096 Buffer Size

8192 Buffer Size

16384 Buffer Size

32768 Buffer Size

65536 Buffer Size

131072 Buffer Size

262144 Buffer Size

524288 Buffer Size

1048576 Buffer Size

2097152 Buffer Size

4194304 Buffer Size

26

Figure 7 Message Polling Stress Test Graph at 0 Intervals

In terms of CPU consumption running the tests on a Pentium 4 on a Dell INSPIRON 5100 with

512MB RAM, results in figure 8 are achieved. As shown below for all test with less than 2MB

buffer size the CPU consumption is minimal and reasonable while it grows to an undesired state

by reaching the 2MB buffer size and reaching even worst situation with 4MB buffer size.

0

500000

1000000

1500000

2000000

2500000

3000000

3500000

0 0.2 0.4 0.6 0.8 1 1.2

# of M

essages Pe

r Second


KM‐UM Message Polling Stress Test

256 Buffer Size

512 Buffer Size

1024 Buffer Size

2048 Buffer Size

4096 Buffer Size

8192 Buffer Size

16384 Buffer Size

32768 Buffer Size

65536 Buffer Size

131072 Buffer Size

262144 Buffer Size

524288 Buffer Size

1048576 Buffer Size

2097152 Buffer Size

4194304 Buffer Size

27

Figure 8 Message Polling CPU Usage Graph

In conclusion the polling algorithm does satisfy the throughput and CPU requirements; however,

as mentioned the response time of the system is a really important factor. As demonstrated by the

results, it is not possible to have any polling interval less than 10 millisecond (5 millisecond on

average) which itself is not a reasonable response time for the vanDisk project. This

communication layer should only be used for passing debugging and non-file system call

messages.

0

20

40

60

80

100

120

0.1 1 10 100 1000

% of C

PU Usage


CPU Usage Diagram

2MB Buffer

4MB Buffer

28

6.0 RECOMMENDATION

My main recommendation to student continuing this part of the vanDisk project is to read my

findings and references to completely understand how KMDF, UMDF and IFS differ. It is really

important to understand the functionalities offered by IFS and take advantage of them.

Furthermore I recommend the future developer of this system to completely read and understand

the minispy project, and later my implementation of the communication layer. By the end of this

process the developer should be comfortable with implementing other functionalities for the

system. I recommend the student continuing this project to read appendix D which explains on

how to setup his environment for easy build and development of the minifilter.

The next step is ideally design and implementation of the communication based on the “push”

algorithm, where the KM driver sends messages to the UM application as soon as they are ready

to be sent. This is done through a blocking send message call by the KM driver. This call

requires the UM application to be expecting the message; thus, it is required to use a separate

thread which is constantly waits for messages from the KM driver and responds to them as

quickly as possible.

Following the completion of the design and implementation of the communication layer, the

replacements for the major operations of the minifilter has to be implemented one by one. One

could do that by implementing a very simple file system initially which simply redirects system

calls at UM level instead of the KM level. Upon a successful implementation of such system, the

common interface has to be implemented which converts the received messages to proper type

for vanDisk and free-loader projects.

In terms of developing the creation of the virtual driver, I recommend the developers to study the

TrueCrypt implementation which contains the complete code for creating and installation of the

virtual drive.

29

7.0 CONCLUSION

It is very important to always consider the expansion of the project and how certain parts of the

project could be re-used. In this experiment I have learned a great deal about different

technologies that support implementation of virtual drives and file system. I have learned that it

is very important to study different options before making any decisions.

Through this report I have concluded that windows shell namespace does not satisfy the needs of

vanDisk project for implementing the required user interface. I have also realized that even

though UMDF and KMDF provide enough functionality; they are not the best solutions as one is

very difficult to implement and debug while the other one is easier but does not include features

offered by IFS.

In conclusion the initial design and implementation of the system that I have placed helps in both

debugging the KM application and understanding and developing the minifilter file system

easier. Finally a major section of my project was focused on understanding the benefits of all

these technologies and their drawbacks. This can be both reused for future project and it is also a

very good starting point for student continuing this project.

30

8.0 APPENDIX A

POLLING INTERVAL

(ms)

256B Buffer Size

(Message/Sec)

64KB Buffer Size

(Message /Sec)

256KB Buffer Size

(Message /Sec)

2MB Buffer Size

(Message /Sec)

4MB Buffer Size

(Message /Sec)0 571363.023 3264616.27 2197973.657 1771971.684 1745715.499 1 199.5125018 51177.55557 204710.2223 1629493.369 1635994.168 2 199.9123265 51177.55557 204710.2223 1631131.051 1635994.168 3 199.7124141 51177.55557 204710.2223 1465725.192 1634356.537 4 199.9123265 51177.55557 204710.2223 1629493.369 1635994.168 5 199.9123265 51177.55557 204710.2223 1627855.688 1635994.168 6 199.9123265 51177.55557 204710.2223 1631131.051 1632722.18 7 199.9123265 51177.55557 204710.2223 1629493.369 1632722.18 8 199.9123265 51177.55557 204710.2223 1632768.733 1632722.18 9 199.9123265 51177.55557 204710.2223 1631131.051 1632722.18 10 199.9123265 51177.55557 204710.2223 1631131.051 1632722.18 20 100.0559195 25614.31539 102457.2615 818840.8891 1090662.779 30 66.77071703 17093.30356 68373.21424 545892.8354 819629.8129 40 50.12781595 12832.72088 51330.88353 409827.4134 656356.9418 50 40.14219524 10276.40198 41105.60792 328516.347 547504.0245 60 33.48471614 8572.087332 34288.34933 274033.3083 469757.0494 70 28.72985778 7354.843591 29419.37436 235120.1097 371150.9158 80 25.16376418 6441.92363 25767.69452 205935.6205 366096.5971 90 22.39018056 5731.886224 22927.5449 183053.8849 329811.1996 100 20.17095383 5163.76418 20655.05672 164910.6325 300122.5593 110 18.35518691 4698.92785 18795.7114 150215.6256 275365.3551 120 16.84082857 4311.252114 17245.00846 137823.3382 254451.7351 130 15.5620063 3983.873612 15935.49445 127356.7261 236490.2461 140 14.46329983 3702.604757 14810.41903 118365.9257 220948.5669 150 13.51287976 3459.297219 13837.18888 110587.4737 206924.1636 160 12.68015328 3246.119239 12984.47696 103772.8664 195342.5873 170 11.94690414 3058.407459 12233.62983 97771.55955 184656.7675 180 11.29326151 2891.074947 11564.29979 92422.70912 175111.3679 190 10.70950384 2741.632982 10966.53193 87645.21946 166538.3285 200 10.18533312 2607.445279 10429.78112 83354.89403 158740.0282 210 9.708242353 2485.310042 9941.24017 79451.10082 151660.2089 220 9.275181288 2374.44641 9497.785639 75907.27792 145206.583 230 8.880492723 2273.406137 9093.624548 72677.18099 139302.4737 240 8.519477983 2180.986364 8723.945455 69722.3946 133883.7543 250 8.188208979 2096.181499 8384.725995 66943.92012 128818.4385 260 7.878201741 2016.819646 8067.278582 64474.6442 124211.3455

31

270 7.596668405 1944.747112 7778.988447 62169.67567 119746.3585 280 7.330713614 1876.662685 7506.65074 59993.68838 115708.8213 290 7.083395969 1813.349368 7253.397472 57970.06641 112054.3951 300 6.852876955 1754.336501 7017.346002 56083.78392 108534.2473 350 5.902829974 1511.124473 6044.497893 48308.38868 93948.17997 400 5.192522767 1329.285828 5317.143314 42494.65186 82835.14777 450 4.631012501 1185.5392 4742.156801 37864.08709 74216.69797 500 4.193960697 1073.653938 4294.615753 34322.60342 67232.63706 550 3.82224716 978.495273 3913.981092 31281.91389 61552.25584 600 3.52433672 902.2302003 3608.920801 28843.08887 56685.07224 700 3.043236787 779.0686174 3116.27447 24906.4753 49064.4924 800 2.688436344 688.2397041 2752.958816 22002.51427 43419.75044 900 2.403945726 615.4101057 2461.640423 19674.90588 39146.42078 1000 2.196836555 562.3901582 2249.560633 17978.50655 35565.09062

Table 4 KM-UM Message Passing Stress Test Result

32

9.0 APPENDIX B

Polling Interval < 2MB Buffer 2MB Buffer 4MB Buffer 0 100 100 100 1 0 2 51 2 0 2 51 3 0 2 51 4 0 2 51 5 0 2 51 6 0 3 51 7 0 2 51 8 0 2 50 9 0 2 51 10 0 2 51 20 0 0 34 30 0 1 25 40 0 0 20 50 0 0 17 60 0 0 14 70 0 0 21 80 0 0 11 90 0 0 10

100 0 0 9 110 0 0 8 120 0 0 7 130 0 0 7 140 0 0 7 150 0 0 6 160 0 0 6 170 0 0 5 180 0 0 5 190 0 0 5 200 0 0 5 210 0 0 4 220 0 0 4 230 0 0 4 240 0 0 4 250 0 0 4 260 0 0 3 270 0 0 3

33

280 0 0 3 290 0 0 3 300 0 0 4 350 0 0 3 400 0 0 2 450 0 0 2 500 0 0 2 550 0 0 1 600 0 0 1 700 0 0 1 800 0 0 1 900 0 0 1

1000 0 0 1 Table 5 KM-UM Message Passing CPU Consumption

34

10.0 APPENDIX C

Major Function Description

IRP_MJ_CREATE The operating system sends an IRP_MJ_CREATE request to open a handle to a file object or device object.

IRP_MJ_CLOSE Receipt of the IRP_MJ_CLOSE request indicates that the reference count on a file object has reached zero

IRP_MJ_READ The IRP_MJ_READ request is sent by the I/O Manager or by a file system driver. This request can be sent, for example, when a user-mode application has called a Microsoft Win32 function such as ReadFile, or when a kernel-mode component has called ZwReadFile Minor Operations: IRP_MN_COMPLETE IRP_MN_COMPLETE_MDL IRP_MN_COMPLETE_MDL_DPC IRP_MN_COMPRESSED IRP_MN_DPC IRP_MN_MDL IRP_MN_MDL_DPC IRP_MN_NORMAL

IRP_MJ_WRITE The IRP_MJ_WRITE request is sent by the I/O Manager or by a file system driver. This request can be sent, for example, when a user-mode application has called a Microsoft Win32 function such as WriteFile or when a kernel-mode component has called ZwWriteFile Minor Operations: IRP_MN_COMPLETE IRP_MN_COMPLETE_MDL IRP_MN_COMPLETE_MDL_DPC IRP_MN_COMPRESSED IRP_MN_DPC IRP_MN_MDL IRP_MN_MDL_DPC IRP_MN_NORMAL

IRP_MJ_QUERY_INFORMATION

This request can be sent, for example, when a user-mode application has called a Microsoft Win32 function such as GetFileInformationByHandle or when a kernel-mode component has called ZwQueryInformationFile

35

IRP_MJ_SET_INFORMATION

It can be sent, for example, when a user-mode application has called a Microsoft Win32 function such as SetEndOfFile or when a kernel-mode component has called ZwSetInformationFile

IRP_MJ_QUERY_EA When a user-mode application has requested information about a file's extended attributes

IRP_MJ_SET_EA The I/O Manager sends the IRP_MJ_SET_EA request to set a file's extended attributes.

IRP_MJ_FLUSH_BUFFERS The IRP_MJ_FLUSH_BUFFERS request is sent by the I/O Manager and other operating system components, as well as other kernel-mode drivers, when buffered data needs to be flushed to disk. It can be sent, for example, when a user-mode application has called a Microsoft Win32 function such as FlushFileBuffers. (For file system drivers and file system filter drivers, calling CcFlushCache is usually preferable to sending an IRP.)

IRP_MJ_QUERY_VOLUME_INFORMATION

The IRP_MJ_QUERY_VOLUME_INFORMATION request is sent by the I/O Manager. It can be sent, for example, when a user-mode application has called a Microsoft Win32 function such as GetDiskFreeSpace or GetFileType.

IRP_MJ_SET_VOLUME_INFORMATION

It can be sent, for example, when a user-mode application has called a Microsoft Win32 function such as SetVolumeLabel.

IRP_MJ_DIRECTORY_CONTROL

It can be sent, for example, when a user-mode application has called a Microsoft Win32 function such as ReadDirectoryChangesW or FindNextVolumeMountPoint or when a kernel-mode component has called ZwQueryDirectoryFile Minor Functions: IRP_MN_NOTIFY_CHANGE_DIRECTORY

Indicates a request for notification of changes to the directory. Usually, instead of satisfying this request immediately, the file system driver holds the IRP in a private queue. When a change occurs to the directory, the file system driver performs the notification, and dequeues and completes the IRP.

IRP_MN_QUERY_DIRECTORY Indicates a directory query request. The types of information that can be queried are file-system-dependent, but generally include the following: FileBothDirectoryInformation FileDirectoryInformation FileFullDirectoryInformation FileIdBothDirectoryInformation FileIdFullDirectoryInformation FileNamesInformation

36

FileObjectIdInformation FileReparsePointInformation

IRP_MJ_FILE_SYSTEM_CONTROL

It can be sent, for example, when a user-mode application has called the Microsoft Win32 DeviceIoControl function to send a file system I/O control (FSCTL) request.

Minor Functions:

File system drivers should handle the following minor function codes: IRP_MN_KERNEL_CALL

This request is the same as IRP_MN_USER_FS_REQUEST (described following), except that the source of the request is a trusted kernel component.

IRP_MN_MOUNT_VOLUME Indicates a volume mount request. If a file system driver receives this IRP for a volume whose format does not match that of the file system, the file system driver should return STATUS_UNRECOGNIZED_VOLUME.

IRP_MN_USER_FS_REQUEST Indicates an FSCTL request, possibly on behalf of a user-mode application that has called the Microsoft Win32 DeviceIoControl function or on behalf of a kernel-mode component that has called ZwDeviceIoControlFile or IoBuildDeviceIoControlRequest. For detailed information about FSCTL requests, see "Device Input and Output Control Codes" in the Microsoft Windows SDK documentation.

IRP_MN_VERIFY_VOLUME Indicates a volume verification request. For removable media, the file system must verify the volume when it detects that the media has been removed and returned to ensure that it is still the same volume that the file system was previously working with. If the volume has changed, the file system should invalidate all outstanding handles. It will also return an error if the file system on this new media has changed. This request is most often used for floppy drives.

File system recognizers must handle the following minor function code: IRP_MN_LOAD_FILE_SYSTEM

Indicates a load-file system request.

IRP_MJ_DEVICE_CONTROL Normally this IRP is sent on behalf of a user-mode application that has called the Microsoft Win32 DeviceIoControl function or on behalf of a kernel-mode component that has called ZwDeviceIoControlFile

37

IRP_MJ_INTERNAL_DEVICE_CONTROL

When Sent The IRP_MJ_INTERNAL_DEVICE_CONTROL request is sent by the I/O Manager and other operating system components, as well as other kernel-mode drivers. Unlike IRP_MJ_DEVICE_CONTROL requests, IRP_MJ_INTERNAL_DEVICE_CONTROL requests are used only for communication among kernel-mode components. While an IRP_MJ_DEVICE_CONTROL request usually originates with a call to DeviceIoControl or ZwDeviceIoControlFile, these routines cannot create IRP_MJ_INTERNAL_DEVICE_CONTROL requests. However, both types of IRP can be created by calling IoBuildDeviceIoControlRequest. Operation: File System Drivers The file system driver should extract and decode the file object to determine whether the request has been issued on a handle that represents a volume open. If this is the case, the file system driver should pass the IRP to the device driver for the storage device on which the volume is mounted. If not, the driver should fail the IRP.

IRP_MJ_SHUTDOWN The IRP_MJ_SHUTDOWN request is sent by the I/O Manager or by a file system driver when the system is being shut down

IRP_MJ_LOCK_CONTROL The file system driver should extract and decode the file object to determine whether the target device object is the file system's control device object. If this is the case, the file system driver should complete the IRP as appropriate without processing the lock request. Otherwise, if the request has been issued on a handle that represents a user file open, the file system driver should perform the o peration indicated by the minor function code and complete the IRP. If not, the driver should fail the IRP. The following are the valid minor function codes: IRP_MN_LOCK

Indicates a byte-range lock request, possibly on behalf of a user-mode application that has called the Microsoft Win32 LockFile function.

IRP_MN_UNLOCK_ALL Indicates a request to release all byte-range locks for a file, usually because the last outstanding handle to a file object is being closed.

IRP_MN_UNLOCK_ALL_BY_KEY Indicates a request to release all byte-range locks with a specified key value.

IRP_MN_UNLOCK_SINGLE Indicates a request to release a single byte-range lock, possibly on behalf of a user-mode application that has called the Microsoft Win32 UnlockFile function.

38

IRP_MJ_CLEANUP When Sent Receipt of the IRP_MJ_CLEANUP request indicates that the handle reference count on a file object has reached zero. (In other words, all handles to the file object have been closed.) Often it is sent when a user-mode application has called the Microsoft Win32 CloseHandle function (or when a kernel-mode driver has called ZwClose) on the last outstanding handle to a file object. It is important to note that when all handles to a file object have been closed, this does not necessarily mean that the file object is no longer being used. System components, such as the Cache Manager and the Memory Manager, might hold outstanding references to the file object. These components can still read to or write from a file, even after an IRP_MJ_CLEANUP request is received.

IRP_MJ_QUERY_SECURITY The IRP_MJ_QUERY_SECURITY request is sent by the I/O Manager. It can be sent, for example, when a user-mode application has called a Microsoft Win32 function such as GetSecurityInfo.

IRP_MJ_SET_SECURITY The IRP_MJ_SET_SECURITY request is sent by the I/O Manager. This request can be sent, for example, when a user-mode application has called a Microsoft Win32 function such as SetSecurityInfo.

IRP_MJ_QUERY_QUOTA When Sent The IRP_MJ_QUERY_QUOTA request is sent by the I/O Manager. This request can be sent, for example, when a user-mode application has called a Microsoft Win32 method such as IDiskQuotaControl::GetQuotaState. Operation: File System Drivers If the file system supports disk quotas, the file system driver should extract and decode the file object to determine whether it represents a user open of a file or directory. If it does, the driver should process the query and complete the IRP. Otherwise, the driver should complete the IRP as appropriate without processing the query.

IRP_MJ_SET_QUOTA When Sent The IRP_MJ_SET_QUOTA request is sent by the I/O Manager. It can be sent, for example, when a user-mode application has called a Microsoft Win32 method such as IDiskQuotaControl::SetQuotaState. Operation: File System Drivers IRP_MJ_SET_QUOTA and IRP_MJ_QUERY_QUOTA existed in Windows NT 4.0 but were not used by file systems. On Windows 2000 and later systems, they are used for disk quota support in NTFS. Support for these IRPs by new file systems is optional.

IRP_MJ_PNP When Sent The Plug and Play Manager sends the IRP_MJ_PNP request

39

whenever Plug and Play activity occurs on the system. Other operating system components, as well as other kernel-mode drivers, can also send certain IRP_MJ_PNP requests, depending on the minor function code. For more information about Plug and Play IRP processing requirements for drivers, see Plug and Play. For reference information about IRP_MJ_PNP minor function codes, see Plug and Play Minor IRPs. Operation: File System Drivers The file system should check the minor function code to determine which operation is requested. File systems must handle the following minor function codes: IRP_MN_CANCEL_REMOVE_DEVICE

Indicates that a previous query-remove device request was canceled. This request is sent to alert the file system in case it needs to perform any cleanup related to the cancellation.

IRP_MN_QUERY_REMOVE_DEVICE Indicates that a device is about to be removed. If a file system is mounted on the device, the PnP Manager sends this request to the file system and to any file system filters. If there are open handles to the device, the file system typically fails the query-remove request. If not, the file system typically locks the volume to prevent future create requests from succeeding. If a mounted file system does not support a query-remove request, the PnP Manager fails the query-remove request for the device.

IRP_MN_REMOVE_DEVICE Indicates that a device is about to be removed. If a file system is mounted on the device, the PnP Manager sends this IRP to the file system and to any file system filters. The file system should immediately pass this IRP to the storage driver for the device, setting a completion routine in which the file system then dismounts the volume.

IRP_MN_START_DEVICE Indicates that a device is being started. The file system should pass this IRP to the storage driver for the device.

IRP_MN_SURPRISE_REMOVAL Indicates that a device has been removed. If a file system was mounted on the device, the PnP Manager sends this IRP to the file system and to any file system filters. The file system should immediately pass this IRP to the storage driver for the device, setting a completion routine in which the file system then dismounts the volume.

Table 6 Mini-Filter Major Functions [10]

40

11.0 APPENDIX D There is a DVD containing the installation files for WDK 6000-16386 that can be requested from

Dr. Ripeanu. In this DVD the files for the vanDisk user interface and FSFilter driver is include in

a directory named “vanDisk”. Within this directory both the report of the project exists and the

files for setting up the developing environment. It is required to get access to Visual Studio .Net

2005 since the project files are stored in the 2005 version format.

This directory contains few batch files in “VanDisk\C\WINDDK\6000\bin” that has to be copied

to the “C:\WINDDK\6000\bin”, also the content of

“VanDisk\C\WINDDK\6000\src\filesys\miniFilter\vanDisk” has to be copied to

“C:\WINDDK\6000\src\filesys\miniFilter\vanDisk”.

Upon acquiring Visual Studio .Net 2005, it is possible to simply open the vanDisk solution file

located at “miniFilter\vanDisk\vanDisk\vanDisk.sln” and build the “filter” that is the KM driver

and the “user”, which is the UM application projects. There is also another project named

“RunMain” which simply runs “vanDisk\vanDisk\RunMain\runbatch.bat” that installs the

vanDisk minifilter.which automatically copies the “minispy.sys” and “minispy.exe” to proper

location and installs the FSFilter driver using the “minispy.inf”. Keep in mind that this batch

doesn’t load the driver.

There are quiet a few batch files developed for easing the process of build and running the

project, following table explains details of each of these batch files and executables:

Batch File Description

vanDisk\copyback.bat Copies the “minispy.exe” and “minispy.sys” to the correct location, and installs the filter using the “minispy.inf” file.

vanDisk\load.bat Loads the minispy filter of the vanDisk project.

vanDisk\unload.bat Unloads the minispy filter of the vanDisk project.

41

vanDisk\show.bat Shows the list of minifilters loaded into the system. This is a quick way to see if the minispy filter is loaded or not.

vanDisk\vanDisk\RunMain\runbatch.bat Runs the “vanDisk\copyback.bat”

vanDisk\vanDisk\Filter\FilterBuild.bat Builds the vanDisk filter project, it requires typing “make” and pressing enter at the end.

vanDisk\vanDisk\User\UserBuild.bat Builds the vanDisk user project, it requires typing “make” and pressing enter at the end.

bin\setenv-vanDisk-filter.bat Setups the build environment for building the vanDisk filter project by only typing “make” in the command line and pressing enter. To clean the project simply type “make /clean” and press enter.

bin\setenv-vanDisk-user.bat Setups the build environment for building the vanDisk user project by only typing “make” in the command line and pressing enter. To clean the project simply type “make /clean” and press enter.

Table 7 Batch File Description

42

12.0 REFERENCES

[1]

[2] J. Yang and F.-B. Sun. “A Comprehensive Review of Hard-Disk Drive Reliability”. In

Proc. of the Annual Reliability and Maintainability Symposium, 1999.

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

armin bahramshahry vancouver, b.c. v7p 3g4 april …armin bahramshahry 1501-2024 fullerton ave....

Documents