winfs

WinFS New Storage Subsystem In Longhorn

John KeefauverManager.NET Business DevelopmentAjilon Consulting

Agenda

User and developer benefits of WinFS

What is WinFS?

WinFS Technology

- Data model

- Schemas

- APIs

- Synchronization

- InfoAgent

- Filesystem

Extending WinFS

WinFS implications

Key Points Integrated storage

Digital-born data

- Business processes generating more digital data

- Personal Memories, e.g. Digital photography and video

Search

- Knowledge workers spending hours daily searching for information

- Industry awaits next leap in productivity

Fortunately, technology supports these trends

- 70%/year individual disk capacity growth over last decade

- Longhorn will have WinFS

User Benefits

Find

- Data is easily found

- Information is organized in the way people think about it

- There is an integrated view of information Relate

- Discovery of data is easy

- Data is smarter and related together Act

- Data flow is controllable

- Preferences for how information is handled

- Work anywhere, anytime, anyplace

WinFS Developer Benefits

New filesystem capabilities

- Current applications continue to work on Longhorn, and benefit from new capabilities

- Metadata and relationships on most file formats

- Metadata handlers for custom file formats

New Windows primitives

- New things in the OS to integrate with

- Extensible to include additional data

New data subsystem capabilities

- Smart connected applications

- Data Sharing

- Branded data to other applications and the Shell

WinFS Is

All end-user data lives in Longhorn

New user experience in Longhorn Shell

A trustworthy place to store data

Data model built on relational database technology

Filesystem capabilities built on NTFS

Everyday Information - domain-specific schemas

Services that make data active

WinFS Is Not

Abandoning legacy applications

Forcing the world to rewrite everything

Complete yet!

- Fundamental model complete for Beta1

- Platform and User Experience complete in Beta2

Just a database

Communication

Avalon WinFS IndigoWindows Forms

ASP.NET

ObjectSpacesObjectSpaces

DataSetDataSet

SQL XMLSQL XML

ProvidersProviders

Fram

ework

ServicesSchemas

Data Model

ADO.NET

Connectivity

Synchronization(WinFS, Win32..)Synchronization(WinFS, Win32..)

InfoAgent(PreferenceRules..)InfoAgent(PreferenceRules..)

FileSystem Services(MetaDataHandlers..)FileSystem Services(MetaDataHandlers..)

CalendarCalendar MediaMedia

DocumentDocument ……

ItemsItems

RelationshipsRelationships

ExtensionsExtensionsCommunications Manager (Port)Communications Manager (Port)

Messaging Services

Transport Channels(IPC, HTTP, TCP…)Transport Channels(IPC, HTTP, TCP…)

Channels (Datagram, Reliable, Peer, …)

Channels (Datagram, Reliable, Peer, …)

Policy EnginePolicy Engine

MessageEncoderMessageEncoder

ChannelSecurityChannelSecurity

QueuingQueuing

EventingEventing

RoutingRouting

TransactionTransaction

Desktop ServicesDesktop Services

Desktop WindowManagerDesktop WindowManager

Presentation Object ManagerPresentation Object Manager

Desktop Composition EngineDesktop Composition Engine

Animation andCompositionAnimation andComposition

Media Services

Hardware RenderingHardware Rendering

MediaProcessingMediaProcessing

Capture and SourcingCapture and Sourcing

Software Renderingand SinksSoftware Renderingand Sinks

Adaptive UIEngineAdaptive UIEngine

Page/SiteCompositionPage/SiteComposition

Personalization andProfiling ServicesPersonalization andProfiling Services

Membership and Security ServicesMembership and Security Services

Designer ServicesDesigner Services

Controls Interop EngineControls Interop Engine

ControlsControls

Windows Forms

ApplicationServicesApplicationServices

PeoplePeople GroupGroup

Collaboration

People andGroupsPeople andGroups

CollaborationHistoryCollaborationHistory

Real-TimeActivitiesReal-TimeActivities

SignalingSignalingFederationFederation

System Services

Transactions Storage

Protocols

Network Services

Kernel M

ode

Base Class LibrariesBase Class Libraries

Memory ManagerMemory Manager

Hosting LayerHosting Layer

Code ExecutionCode Execution LoaderLoader SecuritySecurity SerializationSerialization

LightweightTransactionsLightweightTransactions

TransactionCoordinatorTransactionCoordinator

KernelTransactionManager

KernelTransactionManager

LoggingServiceLoggingService

KernelKernel

Hardware Abstraction LayerHardware Abstraction Layer

ProcessManagerProcessManager

SecurityReference Monitor

SecurityReference Monitor

LPCFacilityLPCFacility

MemoryManagerMemoryManager

PowerManagerPowerManager

ConfigManagerConfigManager

Plug andPlayPlug andPlay

TransactedNTFSTransactedNTFS

Cache ManagerCache

ManagerUniversalData Format

UniversalData Format

FilterEngineFilterEngine

TCP, UDPIPV4, IPV6TCP, UDPIPV4, IPV6 IPSECIPSEC QOSQOS HTTP

ListenerHTTPListener

Internet Connection FirewallInternet Connection Firewall

Demand Activation and Protocol HealthDemand Activation and Protocol Health

PNRPPNRP NativeWiFiNativeWiFi SIPSIP TCP

ListenerTCP Listener

UDP ListenerUDP Listener

IPC ListenerIPC Listener

Network Class LibraryNetwork Class Library

GDI/GDI+GDI/GDI+ WindowManagerWindowManager

GlobalAudio Engine

GlobalAudio Engine

Direct 3DGraphicsDirect 3DGraphics

Graphics driversGraphics drivers

DDIDDI InputManagerInputManager

AudioDriversAudioDrivers

DirectX GraphicsMini port

DirectX GraphicsMini port

RedirectorsRedirectors

SCSI/FCSCSI/FC 802.3802.3 802.11802.11 ....

Device Drivers

ManagementServices

(Event Logs,Tracing,Probes,

Auto Update,Admin)

ManagementServices

(Event Logs,Tracing,Probes,

Auto Update,Admin)

IO ManagerIO Manager

Application DeploymentEngine (Click-Once)

Application DeploymentEngine (Click-Once)

Identity &SecuritySystem

Identity &SecuritySystem

FAT 16/32FAT 16/32

FilterManagerFilterManager

DistributedFile SystemDistributedFile System

Virtual ShadowCopy ServiceVirtual ShadowCopy Service

File ReplicationServiceFile ReplicationService

Virtual DiskServiceVirtual DiskService

Models

ServiceServiceObjectObject T/SQLT/SQL XMLXMLDocumentDocument UIUI MediaMedia

CLR

Presentation Data Communication

Base Operating System Services

Avalon WinFS Indigo

WinFS Overview

Complete Data SubSystem- Object, Relational, and XML

Schema Framework- Extensible “everyday info” schemas- ISV-built schemas

Data store and Model - Items- Relationships- Extensions

Data Services- Synchronization- InfoAgent- Filesystem Services

NTFS- NTFS Semantics

for Filestreams- Backwards Compatible

with Win32

Complete Data SubSystem- Object, Relational, and XML

Schema Framework- Extensible “everyday info” schemas- ISV-built schemas

Data store and Model - Items- Relationships- Extensions

Data Services- Synchronization- InfoAgent- Filesystem Services

NTFS- NTFS Semantics

for Filestreams- Backwards Compatible

with Win32

WinFS Data Model

Items

- The new atomic unit of data Items have subsumed Files Copy, put in Folders, etc.

- A group of simple and complex types that represent data Defined in a schema, arranged

in types

- Structured or semi-structured

- Persisted

Relationships- Explicitly relate Items together

E.g.; Author binds Document to Contact

- Schema can model complex items- Containment, reference, embedding,

categories, etc. Extensions

- Provide ability to add new data to existing Item types

Fram

ework

Models

Core WinFS ItemsItems


ExtensionsExtensionsFilesystem Srvcs (Handlers, …)Filesystem Srvcs (Handlers, …)

OperationsOperations

Data Model

NTFSNTFS

Relational EngineRelational Engine

ServicesPeoplePeople

DocumentsDocuments

……InfoAgent (Rules, …)InfoAgent (Rules, …)

Synchronization(WinFS, …)Synchronization(WinFS, …)

Schemas

XMLXMLAPIs

T/SQLT/SQLObjectsObjects

Data Model Goals

Mechanism to schematically model complex items

- Documents, pictures, people, video, sound, etc.

- Provide mechanism to implement complex item behavior

- Embedding links to model compound items

OLE Document Files

Mail messages and attachments

Provide organizational facilities for items

- Relationships as first class entities to relate items

- Queries as way to retrieve and classify items

- Folders as a mechanism for explicitly organize items

WinFS Schemas

Windows Everyday Information- Documents, Messages,

Annotations, Notes- Media, Audio, Video, Images- Events, Appointments,

Locations, UserTask Windows System

- SystemTasks, Config, Programs- Explorer, Help, Security

New Schemas- Developers can define own

data shape comprised of Scalars Complex Types XML Binary/Filestream

Fram

ework

Models





Data Model

NTFSNTFS



DocumentsDocuments



Schemas

XMLXMLAPIs


WinFS APIs

Objects- VB, C#, etc. strongly

typed classes- The way to program to WinFS- Lots of WinFS helper functions- System.Search

T/SQL- ADO.NET- Standard relational language for

complex queries- Enables view integration with other

data sources- Read only

XML- WinFS Items can be serialized into

XML- Enables data interop

Fram

ework

Models





Data Model

NTFSNTFS



DocumentsDocuments



Schemas

XMLXMLAPIs


API Examples

VB Managed API

Dim personItem As Person

For Each personItem In Person.FindAll(context, “PersonalNames.Surname=’Smith’)

...

Next

• T-SQLselect p._Item from [System.Storage.Contacts.Store].

[Person] p where exists (select * from unnest

(p.PersonalNames) n where n.Surname=‘Smith')

WinFS ServicesFilesystem

“File-backed” Items

- Items with traditional filestream parts within

- Uses real NTFS streams and file handles

- Any file can be imported into WinFS as a File-back Item

WinFS is backwards compatible with Win32

Fram

ework

Models





Data Model

NTFSNTFS



DocumentsDocuments



Schemas

XMLXMLAPIs


Longhorn And Filesystems

Files can live solely in an NTFS volume

- Available for boot E.g., C:\Windows is in NTFS

- Volume can be mounted on down level machine E.g., Firewire drive on both XP and Longhorn

Items can live solely in WinFS

- File-backed Items Accessible through standard Win32 APIs

Metadata Handlers get data in and out of file streams

- User data moved into WinFS I.e., C:\Documents and Settings

- Has Import/Export utilities

WinFS ServicesOperations

Security Config

Manageability

Backup/Restore

App install

- Schemas, classes, replication adaptors, preference rules, etc.

- Versioning

Snapshot/TimeWarp

Anti Virus

Quota

Admin Tools

Import/Export

Fram

ework

Models





Data Model

NTFSNTFS



DocumentsDocuments



Schemas

XMLXMLAPIs


WinFS ServicesSynchronization

Synchronize one WinFS with another

- Keep My Contacts and My Files in sync across my home machines

- Peer to Peer sharing

Synchronize WinFS with other data sources

- Keep My Contacts in sync with online email contacts, enterprise CRM, etc.

Fram

ework

Models





Data Model

NTFSNTFS



DocumentsDocuments



Schemas

XMLXMLAPIs


WinFS ServicesInfoAgent

Users want to control how their PCs behave

- It’s called a personal computer after all

- Every aspect of the system can be personalized

InfoAgent enables rich, flexible customization

- “When I receive a high priority email from a customer, show me a popup message if I’m at my desk, otherwise forward it to my cell phone”

- “When I download new photos from my camera, relates them to the events on my calendar”

Fram

ework

Models





Data Model

NTFSNTFS



DocumentsDocuments



Schemas

XMLXMLAPIs


Notifications And InfoAgent

‘Active Data’ – Subscribe to WinFS changes

Item change subscriptions

Item Domain containment/query subscriptions

InfoAgent Integration

Inclusive set of events, contexts, and actions

Preferences stored as WinFS items

Unified management of notification rules

ActionsActionsPreferencesPreferences

EventsEvents

ContextsContexts

Fram

ework

Models





Data Model

NTFSNTFS



DocumentsDocuments

……InfoAgent(Rules, …)InfoAgent(Rules, …)


Schemas

XMLXMLAPIs


WinFS Extensibility Points

11

22

33

44

55

Extend and Relate Windows types

Build your own Types

Build Metadata handlers for

custom file formats

Build Synchronization adaptors and custom resolution handlers

Build on InfoAgent

66 Integration with Longhorn Shell

11

22

33

44

55

66

WinFSWhat does this mean?

The new way to think about data

- Being a Data Geek not required!

Exposes data in a way that makes a user’s data more useful to them

New managed APIs for new entities in Windows, and for new data

An storage subsystem layer enabling you to do more and do it quickly and easily

A significant leap forward that’s 100% backwards compatible

winfs

Business

data data model

winfs data model items

branded data

discovery of data

data flow

easy data

data active

schemas data store