sql server notifications: invalidating cached results michael pizzo software architect microsoft...
DESCRIPTION
Tools Client Application Model AvalonWindows Forms Web & Service Application Model ASP.NET / Indigo Win FS Compact Framework Yukon Mobile PC Optimized System.Help System.Drawing System.NaturalLanguageServices Data Systems Application Model Presentation Data Mobile PC & Devices Application Model Communication Command Line NT Service DataSet Mapping ObjectSpaces ObjectSpace Query Schema Item Relationship Media Audio Video Images System.Messaging System. Discovery System.DirectoryServices System.Remoting System.Runtime.Remoting Active Directory Uddi System.Web.Services Web.Service Description Discovery Protocols System.MessageBus Transport Port Channel Service Queue PubSub Router System.Timers System.Globalization System.Serialization System.Threading System.Text System.Design Base & Application Services Fundamentals System.ComponentModel System.CodeDom System.Reflection System.EnterpriseServices System.Transactions Security System.Windows. TrustManagement System.Web. Security System.Message Bus.Security Authorization AccessControl Credentials Cryptography System.Web.Configuration System.MessageBus.Configuration System.Configuration System.Resources System.Management System.Deployment System.Diagnostics ConfigurationDeployment/Management System.Windows System.Windows.Forms System.Console System.ServiceProcess System.Windows.Forms System.Web System.Storage System.Data.SqlServer Animation Controls Control Design Panel Controls Dialogs SideBar Notification System.Windows Documents Text Element Shapes Shape Ink UI Element Explorer Media System.Windows.Forms Forms Control Print Dialog Design System.Web.UI Page Control HtmlControls MobileControls WebControls Adaptors Design Ports InteropServices System.Runtime System.IO System.Collections Generic System.Search Annotations Monitoring Logging Relevance System.Data SqlClient SqlTypes SqlXML OdbcClient OleDbClient OracleClient Core Contact Location Message Document Event System.Storage System.Web Personalization Caching SessionState System.Xml Schema Serialization Xpath Query Permissions Policy Principal Token System.Security System.Collaboration RealTimeEndpoint TransientDataSession SignalingSession Media Activities HttpWebRequest FtpWebListener SslClientStream WebClient System.Net NetworkInformation Sockets Cache System.Web Administration Management Navigation Peer Group Policy Serialization CompilerServices Recognition System.Speech SynthesisTRANSCRIPT
SQL Server Notifications: Invalidating Cached ResultsMichael PizzoSoftware ArchitectMicrosoft [email protected]
Session Code: Dat405
AgendaBuilding a Scalable EnterpriseWhidbey SqlNotifications and CachingImplementingSummary
Tools
Client Application Model
Avalon Windows Forms
Web & Service Application Model
ASP.NET / Indigo Win FSCompact
FrameworkYukon Mobile PC Optimized
System.Help
System.Drawing
System.NaturalLanguageServices
Data Systems Application Model
Presentation Data
Mobile PC & Devices Application Model
Communication
Command Line
NT Service
DataSet
Mapping
ObjectSpaces
ObjectSpace
Query
Schema
Item
Relationship
Media
Audio
Video
Images
System.Messaging System.Discovery
System.DirectoryServices
System.Remoting
System.Runtime.Remoting
ActiveDirectory
Uddi
System.Web.Services
Web.Service
Description
Discovery
Protocols
System.MessageBus
TransportPort
Channel
Service
QueuePubSub
Router
System.Timers
System.Globalization
System.Serialization
System.Threading
System.Text
System.Design
Base & Application Services
Fundamentals
System.ComponentModel
System.CodeDom
System.Reflection
System.EnterpriseServices
System.Transactions
SecuritySystem.Windows.TrustManagement
System.Web.Security
System.MessageBus.Security
Authorization
AccessControl
Credentials
Cryptography
System.Web.Configuration
System.MessageBus.Configuration
System.Configuration
System.ResourcesSystem.Management
System.Deployment
System.Diagnostics
Configuration Deployment/Management
System.Windows System.WindowsSystem.Windows.Forms
System.Console
System.ServiceProcess
System.Windows.Forms System.Web System.Storage System.Data.SqlServ
er
Animation
Controls
Control
Design
Panel
Controls
Dialogs
SideBar
Notification
System.Windows
Documents
Text Element
Shapes
Shape
Ink
UI Element Explorer Media
System.Windows.Forms
Forms
Control
Print Dialog
Design
System.Web.UI
Page
Control
HtmlControls
MobileControls
WebControls
Adaptors
Design
Ports
InteropServices
System.Runtime
System.IO
System.Collections
Generic
System.Search
Annotations
Monitoring
Logging
Relevance
System.Data
SqlClient
SqlTypes
SqlXML
OdbcClient
OleDbClient
OracleClient
Core
Contact
Location
Message
Document
Event
System.Storage
System.Web
Personalization
Caching
SessionState
System.Xml
Schema
Serialization
Xpath
Query
Permissions
Policy
Principal
Token
System.Security
System.Collaboration
RealTimeEndpoint
TransientDataSession
SignalingSession
Media
Activities
HttpWebRequest
FtpWebListener
SslClientStream
WebClient
System.Net
NetworkInformation
Sockets
Cache
System.WebAdministration
Management
Navigation
Peer Group
Policy
Serialization
CompilerServices
Recognition
System.Speech
Synthesis
AgendaBuilding a Scalable Enterprise
Scalable Data-Driven Enterprise ApplicationsScaling SolutionsWeb Caching BenefitsCaching in ASP.NET 1.xCaching Dynamic Content
Whidbey SqlNotifications and Caching Implementing Summary
Scaleable Data-Driven Enterprise Applications
Data-Driven Enterprise ApplicationsWebServicesDynamic Web Applications
Product-driven storefrontSearch sitesInformational sites (weather, traffic, news, …)PortalsInternal lookup tables (zip codes, tax tables,…)Etc…
Increased Scalability==Reduced Contention for shared resources
In Data-Driven applications, data becomes the shared state that limits scalability
Scaling SolutionsPartitioning
Scale out by partitioning the data across multiple serversRoute request to the appropriate serverAppropriate when:
Results can be served from a single partitionAppropriate server can be easily identified
Replication CachingNormalized view is most appropriateFew results are common, so base data is candidateBase data too costly to invalidate, and must be maintainedRich query support required
Web Application CacheTransformations/processing between normalized data in Database and objects in cacheMany results (or candidate results) are commonConstruction of results are not too costly to be invalidatedCached objects can be adequately queried with ADO.NET or application code
These types of caching are highly Complementary
Web Caching BenefitsImprove Response Time
Reduce Code ExecutionReduce process hopsReduce managed/unmanaged transitionsReduce Kernel/User mode transitionsReduce database round-tripsEliminate server hops
Improve ScalabilityDatabase is generally bottleneck
Shared stateReduce number of requests to Database
Caching In ASP.NET 1.xObject Caching
ASP.NET Applications can add/remove objects from cache through dictionary functions or Cache APIs
DataSet ds = (DataSet)Cache[ProductID]; if (null == ds)
{ ds = new DataSet(); SqlDataAdapter adapt = GetDataAdapter(); adapt.Fill(ds);
Cache.Insert(ProductID, ds); }
Response CachingPage Directives can instruct ASP.NET to cache responses
<%@ OutputCache Duration="3600" VaryByParam="CategoryId;selection" %>
Fragment CachingPage Directives can instruct ASP.NET to cache values from individual controls
Entries invalidated based on Time (TTL, absolute time, time unused), File-event, or removal of other entries
Caching Dynamic ContentDelay in publishing new Content
i.e., new Products, CategoriesDelay in returning new information
i.e., from a WebServiceInconsistent results across WebFarm
Refreshed at different intervalsNo ability to invalidate
i.e., when prices change/invalid informationUsers not seeing their changes
i.e., configuration settings in a PortalWorkarounds
Coordinated absolute time expirationData that can accept latency
Explicit invalidationsWhere application controls changes to data
Agenda Building a Scalable Enterprise
Whidbey SqlNotifications and CachingSQL Server Query Processor NotificationsReceiving NotificationsObject Caching in ASP.NETResult Caching in ASP.NET
Implementing Summary
SQL Server QP Notifications
User Requests Notifications on QuerySets as command properties
Specifies SQL Service Broker queueSpecifies message to post on change
QP generates a “notification template”Supports common Selects, Stored procs, Batches containing notifiable queriesSupports same queries as indexed views
Must be deterministicMust not be overly complex
Well suited for small number of queries with many parameter valuesNOT Designed for large number of ad-hoc queries
Posts to SQL Service Broker when results changeDurable; doesn't require continuous connectionProvides activation model for processing message
Supported in SQL Server “Yukon”
Using QP Query Notifications
Create a CommandCreate a SqlNotificationRequest
Specify message, message queue, timeoutAssociate the SqlNotificationRequest with one or more commandsExecute the command(s)
Execution registers for notificationsUse ExecuteNonQuery to register without retrieving results
Query SSB message queue for your message
Poll or blocking “waitfor” querySynchronous or Asynchronous
Application
Query Processor Notifications
SSB
Client Machine Database Server
SqlNotificationRequest
SqlCommandTemplat
e
Execute()Execute()
SqlCommand Execute()Execute()
Products
SqlDependencySqlDependency uses QP Notifications to raise a client side event when results change
Registers with a runtime listener upon creationListens on HTTP or TCP port
Can be associated with one or more commandsCreates a SqlNotificationRequest under the coversSpecifies routing message and default message queue
Exposes an OnChanged EventNotification Dispatcher fires notification from database
Activated upon delivery of message to queueUses information in message to determine destination
Using SqlDependencyCreate a SqlDependencyRegister for the OnChanged eventAssociate SqlDependency with one or more commandsExecute the command(s)Event raised when results changepublic DataSet getData() public DataSet getData() {{ SqlDependency depend = new SqlDependency();SqlDependency depend = new SqlDependency(); depend.OnChanged += new EventHandler(ChangeEventHandler);depend.OnChanged += new EventHandler(ChangeEventHandler); ds = new DataSet();ds = new DataSet(); SqlDataAdapter adapt = getDataAdapter();SqlDataAdapter adapt = getDataAdapter(); depend.AddCommandDependency(adapt.SelectCommand);depend.AddCommandDependency(adapt.SelectCommand); adapt.Fill(ds);adapt.Fill(ds); return ds;return ds;}}static void ChangeEventHandler(Object Sender, EventArgs e) {static void ChangeEventHandler(Object Sender, EventArgs e) {
MessageBox.Show("Results Changed");MessageBox.Show("Results Changed");}}
Query Processor Notifications
NotificationDeliveryService
SSBApplication
DependencyListener
Client Machine Database Server
SqlCommand
SqlDependency
Template
OnChangeAddCommandDependency()AddCommandDependency()
Execute()Execute()
Invalidate(ID)Invalidate(ID)
SqlNotificationRequestProducts
SqlDependency
Object Caching With Data Invalidations
Objects placed in ASP.NET Object Cache can be dependent on database resultsUse new SqlCacheDependency
Supports QP NotificationsPass command into constructor
Creates a SqlDependency under the coversAlso supports polling for pre-Yukon databases
Specify database and tablename in constructorDatabase and tables must be configured to support
listener tables, triggers and stored proceduresCaching information added to web server configuration fileObject invalidated when any change is made to the dependent tableSee WS330 for more information
Using Object Caching With Data Invalidations
Generate the object to be cachedAssociate a SqlCacheDependency with each command (or table) used to generate the objectExecute the command(s) to generate the response
Add the object to the cache, specifying the SqlCacheDependency
DataSet ds = (DataSet)Cache[ProductID]; if (null==ds) { ds = new DataSet(); SqlDataAdapter adapt = getDataAdapter(); SqlCacheDependency cacheDep =
new SqlCacheDependency(adapt.SelectCommand); adapt.Fill(ds); Cache.Insert(ProductID, ds, cacheDep);}
Response Caching With Data InvalidationsASP.NET Responses can be dependent on database
resultsExplicitly associate SqlCacheDependency(s) with ResponseOr Specify as part of caching page directive
For query notifications, specify:SqlDependency=“CommandNotification”SqlDependency=“CommandNotification”
ASP.NET generates a SqlDependency, associates it with the ASP.NET generates a SqlDependency, associates it with the response, and adds the dependency to the application response, and adds the dependency to the application contextcontextCommands automatically enlist in contextCommands automatically enlist in context
For table dependencies, specify:SqlDependency=“database:tablename”SqlDependency=“database:tablename”
When used with II6.0, responses are pushed to Kernel Mode Response cache!
Using Response Caching With Data invalidations
Set up the dependency manuallySpecify that the page is to be cachedGenerate the page
Create a SqlCacheDependency for each command used to generate the response
Add the SqlCacheDependency(s) to the Response
<%@ OutputCache Duration=3600 VaryByParam=“none”<%@ OutputCache Duration=3600 VaryByParam=“none”SqlDependency=“CommandNotification" %>SqlDependency=“CommandNotification" %>
Response.Cache.SetExpires(DateTime.MaxValue);Response.Cache.SetExpires(DateTime.MaxValue);SqlDataAdapter adapt = getDataAdapter();SqlDataAdapter adapt = getDataAdapter();SqlCacheDependency cacheDep = SqlCacheDependency cacheDep =
new SqlCacheDependency(adapt.SelectCommand);new SqlCacheDependency(adapt.SelectCommand);adapt.Fill(ds);adapt.Fill(ds);Response.AddCacheDependency(cacheDep);Response.AddCacheDependency(cacheDep);
Or use the outputcache directiveSpecify: SqlDependency="CommandNotification" Set Command.NotificationAutoEnlist to false for any
commands you don’t want to notify on
ASP.NET Application/ASP.NET Application/WebServiceWebService
<%<%@ OutputCache @ OutputCache Duration=3600 Duration=3600 VaryByParam=“none”VaryByParam=“none”SqlDependency= SqlDependency= “CommandNotification” “CommandNotification” %>%>
Response Caching In ASP.NET
NotificationDeliveryService SSB
Http.Sys
Middle Tier Database Server
SqlCommand
ChangeDetectionExecute()Execute()
Invalidate(ID)Invalidate(ID)
ResponseCache
SqlCommand Execute()Execute()
IIS
Products
Web Application/ WebService Data Caching
Agenda Building a Scalable Enterprise Whidbey SqlNotifications and Caching
ImplementingDeciding What to CacheSqlClient Notification Support
Summary
Deciding What To CacheData Caching
Cache queryable data (i.e., DataSet) in Object CacheAppropriate when some subset of data is commonly useful across requests, objects
Object CachingCache common objects based on resultsAppropriate when same object may be used by multiple requestsAdded benefit if object creation/transformations is expensive
HTTP Response CachingSpecify that entire Response should be cachedAppropriate when entire response is common to multiple users
Fragment (Partial Page) CachingSpecify that some subset of response is to be cachedAppropriate when a subset of response is common, but also contains dynamic content
SqlClient Notification Support
New SqlNotificationRequest ObjectUsed to specify Message, Message Queue, Timeout
New SqlDependency ObjectAddCommandDependency() Method
Sets Notification information on a CommandOnChanged Event
Raised when any results changeDelivery properties
Protocol, Port, Timeout,…New SqlCommand Properties
New Notification PropertyUsed for setting SqlNotificationRequest object
NotificationAutoEnlist PropertyAllows SqlCommand not to be enlisted in Notification Context
Summary Of Microsoft Web Caching Architecture
Developer has a choice!Cache transformed data or objects in application cacheEasy to use
SqlDependency, ASP.NET Page DirectivesExplicit control
Use Page DirectivesEnlist commands by default
Specify commands that shouldn’t be enlistedSpecify dependent tables
Support for pre-Yukon ServersCreate explicit SqlCacheDependenciesCreate SqlDependencies
Specify Transport, Port, alternate message queueAssociate with individual command(s)
Use SqlNotificationRequestSpecify custom message queueQuery queue directly using blocking or unblocking, synchronous or asynchronous queries
Questions?
SQL Server ResourcesWeek long SQL Server “Ask the Experts” lounge in:
Foyer outside Room 309Support for SQL Server “Yukon” PDC Preview at SQLJunkies
http://www.sqljunkies.com/forumsSQL Server “Yukon” FAQ Blog at SQLTeam
http://yukonblog.sqlteam.com/
Other Key Resourceshttp://www.microsoft.com/sql/communityhttp://msdn.microsoft.com/sqlserver/34 world wide user groups,
http://msdn.microsoft.com/usergroups/find.aspOur most active SQL Server newsgroups,
Microsoft.public.sqlserver.programmingMicrosoft.public.sqlserver.serverMicrosoft.public.sqlserver.dtsMicrosoft.public.sqlserver.olapMicrosoft.public.sqlserver.setupMicrosoft.public.sqlserver.replicationMicrosoft.public.sqlserver.msde
© 2003-2004 Microsoft Corporation. All rights reserved.This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.