caching: how it works and when you should use it shy cohen principal, shy cohen consulting aap314
TRANSCRIPT
Caching:How It Works and When You Should Use It Shy CohenPrincipal, Shy Cohen Consulting
AAP314
2
What Gurus Need to Know
What’s a cache and what it doesCore and advanced conceptsCaching on premises and in Windows Azure
Goal: Grok CachingReally, really understand it!Become caching gurus
Non-goal: DemosDownloadSamplesVideos
Concepts Guidance
3
What is Caching?
A. A mechanism for storing data close to where it’s used
B. A mechanism used to speed up a systemC. A mechanism for decreasing load on a system
componentD. All of the above
Not a new concept
4
The Cardinal Rule of Caching
In order to use caching, your app needs to be OK working with possibly stale data
Possibly different than what’s stored in the System of Record
Cached data should be considered “historical”Captured at some point in the pastMay not represent current state
There are ways to overcome thisUsing specific patterns, which we’ll cover in this talk
5
Example: Caching in a Web Application
CacheCache
Speeds up the
application
Reduces database load
Shared
CacheV1 V2
Update
Same dataacross all servers
Load Balancer
High load on the
database
Long load time & delays
Users see different data, depending on the server
they get routed to!
6
Windows Server AppFabric Caching
A free, distributed, in-memory shared cacheCombines the memory capacity of multiple cache servers to create a large cache
A place for applications to temporarily store information they use often
The cache holds key-value pairsCan hold any (serializable) CLR Object
Broader scoped than on-machine cachingMore memory, more users, more possibilitiesBetter overall performance & system utilization
Basic Concepts
CacheConcepts
8
Unified View
Nodes are coordinated to create a unified view of the cache for the cache clients
Cluster is abstracted by the APINo need to know on which node the data is or where it should be stored
Client’s View
Cache Segment
#3
Cache Segment
#2
Cache Segment
#1
CacheConcepts
9
Data Categorization
Reference Data“Read-Heavy”, Shared between users (concurrent access)Examples: Product description, customer profile data
Resource DataRead-Write, Shared between users (concurrent access)Examples: Number of units in stock, online survey results
Activity DataRead-Write, User specific (no concurrent access)Examples: Shopping cart content, survey response
10
Caching Guru Pop-Quiz
NameData
Affinity
Access Patter
n
Shared“Read-Heavy”
SharedRead-Write
User Specific
Read-Write
Reference
Resource
Activity
CacheGuidance
11
Cache
Determine What Data to Cache
Categorize your data and cache appropriately
Affects where to store, how long to store for, whether you want explicit deletion, etc.
Cache the right dataOutput is not always the most efficient thing to storeInstead, consider storing independently-meaningful sub-sets that can be used independently
User Settings Service
Company Settings
User Customizatio
n
C1 U1
Caller
C1+U1
U2
C1
C1+U2
Get Settings for User 1 of Company 1
Get Settings for User 2 of Company 1
C1+U1
C1+U2
CacheGuidance
12
Assume “Best Effort” Caching
Application must withstand cache being unavailableHardware failure, configuration problems, upgrades
Application must withstand cached data unavailability
Do not trust information to stay in the cacheCache doesn’t guarantee data retention
Do not store “important” information only in the cache
Costly to reproduceCrucial for correct operation of your application
CacheGuidance
13
Use an Effective Caching Strategy
Carefully balance cached objects’ size vs. number of objects
Performance degrades when objects are largeCache the smallest objects possiblePerformance also affected by number of the round-trips you need to make to the cache clusterMeasure performance & optimize size/number for your scenario
Keep data coherentDo not break down a large object to multiple small ones since you may not be able to reconstruct it
CacheGuidance
14
Craft Your Keys Wisely
Use the minimal but compete set of unique identifiers for the data you’re caching
Keys consume space in the cache
CacheConcepts
15
Write-Aside Cache
Provides a place for the application tostore data “nearby” for quick accessClient is responsible for managing the content
Storing, fetching, updating, deleting*Client can optionally specify cached objects’ lifetime
Optionally changing the cluster’s default expiration policyOptionally set per-item expiration when adding dataOptionally update the expiration time of a cached objectThe cluster will auto-remove objects based on these settings
Cache
Service / Website
Data Sourc
e
Caller
CacheGuidance
16
Use Write-Aside Cache Appropriately
Use it for Reference DataPractically no data access contentionBut there’s a more efficient solutions (details soon)
Use it for Resource DataOnly with the appropriate concurrency control mechanism in place (details soon)
Use it for Activity DataPerfect fit, since there’s no data access contention
CacheConcepts
17
Regions
“Name areas” within the cacheThere’s always a default one (called “default”)You can programmatically create and delete regions
Useful when multiple apps share a cache clusterAllows you to avoid key name-collisionsCan also be used for different areas within a single app
Cache Cluster “default” “Permissions” “Last Viewed Items”“Friends’
Purchases”
CacheConcepts
18
Tags
Metadata to describe an object in the cacheA string, encapsulated in a DataCacheTag instanceItems can have zero, one, or more tags
Tags are specified when the object is createdCan’t update or remove tags for existing objectsTIP: Replace the entire object to update its tags
You can search the cache based on these tagsAn API lets you find objects based on one tag or multiple tags (“any” or “all”)
Basic Concepts - Review
Unified ViewData CategorizationWrite-Aside CacheTags & Regions
Auto Object Removal
CacheConcepts
21
Object Removal
Explicit DeletionUser explicitly removes an item from the cache
ExpirationWhen an object is proactively removed from the cache because of staleness
EvictionWhen an object is reactively removed from the cache due to low memory conditions
22
Specifying Expiration
Expiration is a hint to the system on how long to keep an object in the cache
“I won’t need this object after X minutes”Cache will not return expired objects, but might keep them in memory
ConfigurableFor the cache cluster, orAt each Add/Put call
Relative to when the object was addedA special technique allows exact expiration as well (details soon)
Can extend the expiration period for existing cached objects
23
Empty Space
Expiration and Eviction Thresholds
Cached Data
Cached Data
Empty Space
Empty Space
90%
70%
Non ExpiredObjects Evicted
Expired Objects Evicted
Cached Data
High Watermark
Low Watermark
Eviction can be turned off.Use with great
care!
CacheGuidance
24
TIP: Local-Time Based “Expiration”
Enable expiry at a certain (predetermined) timeRelative to the client’s time-zone
Create a function to map “Now” to a time segmentThis hour, Today, This week
Add the function’s result to the key you’re generatingKey = original key + suffixE.g. Product Price Cache Key = SKU + Today’s Date
Use the key to store the data in the cacheWhen “Now” maps to a different time segment
The generate key will no longer match the key used for storingTherefore, you will no longer “find” the object in the cache
Local Cache
CacheConcepts
26
Client-Side Caching
Essentially, another layer of cachingGreat for Reference DataMay not be suitable for other types of data
When enabled, client stores a localreference to a deserialized version offetched cache-objects
Stored in the memory of the client applicationSaves network access time and deserialization
On subsequent requests for a previously fetched object, it returns a reference to the locally cached one
Without contacting the server
CacheService
/ Website
Data Source
Caller
27
Object Removal from the Local Cache
Local cache is more constrained then the cache cluster
Configurable limit on the number of objects stored locallyDefault is 10,000 objects
Local cache removal policy is based onMax objects: when reached, the local client will attempt to remove ~20% of the LRU locally cached objectsLocal lifetime restrictions: configured per clientPull-based notifications: optional pull for cache changes
With a per-client configurable pull interval
CacheGuidance
28
Use Local Cache With Care
Local caching increases memory pressure on the client
This can actually make the application run slower
Pull-notifications are not cost-freeLocal cache doesn’t make copies of the objects
I.e. all local callers get same instance of the locally cached objectExercise care when changing the object from multiple places
The 2nd layer of caching increases stalenessUse it for client-side immutable objects that change infrequently on the server
Concurrency
CacheConcepts
30
Data Item Versions
Every Cached items has a versionVersion is auto-assigned by the cache
Version can be used forSaving bandwidth by only fetching changed items
GetIfNewer takes the current version you have
Optimistic concurrencyVersion-dependent overloads for Put and Remove
CacheGuidance
31
Fetch Only When Cache Data Changed
Specify last fetched version when asking for an object from the cache
Remember the objects you fetched, and their version numbersRe-fetch objects only if they changed
Guidance only applies for non-trivial objectsMight not be worth the effort for very small objectsWill depend on your specific configurationMeasure to find out your specific size threshold
CacheConcepts
32
Optimistic and Pessimistic Concurrency
Optimistic concurrencyManaged through versionsMost apps don’t have conflict scenariosBut resolving conflicts may not be trivial
Pessimistic concurrencyUse when concerned about the app making concurrent updatesLock based. Only one client can own the lock at a timeGetAndLock returns a DataCacheLockHandle PutAndUnlock requires a DataCacheLockHandle
CacheGuidance
33
Use Locking API Carefully
Locking is not enforcedI’s an opt-in modelAll users must “play nice” in order for this to work
Use regions and a naming convention to help developers know they need to use lockingLocked objects won’t expire, but they may expire immediately if unlocked after their expiration time
TIP: PutAndUnlock has overloads to extend the expiration period
Advanced Concepts
35
The Case for Cache Data-Protection
Easy to recreate data In-memory cache only“Important” data In-memory cache, backed by persistent storage (e.g. SQL)Something in the middle ?
Data that you’d prefer not to loseData that’s really expensive to produceExamples: User session state, Mainframe data
CacheConcepts
36
High Availability
Provides cache data-protectionImplemented by saving 2 copies of the cached data on different nodes in the cluster
Safeguards against cache node failuresUpon node failure, single-copy data is replicatedDoesn’t protect against more than one node failing
CacheGuidance
37
Verify Your Need for HA
By default, your app should withstand cache data lossHigh-availability costs
Performance is lowered due to replicationHA halves your available memory
Consider setting up separate cache clusters for high-availability scenarios
CacheConcepts
38
Graceful Shutdown
Added in AppFabric Caching v1.1Moves node data to other cluster nodes before shutting a node downUseful for planned upgrades, patches, etc.
Relevant for both HA and non-HA scenarios
39
The Case for Cache-Source Coherency
To maintain cache-source data coherency, every the cache needs updating every time the source is updated
Can we enforce that?Read involves 2 or 6 network “legs”
Can we make this more efficient?We have 2 data access concepts
Name-value pairs for the cacheSomething else for the data sourceCan we unify this?
Cache
CacheConcepts
40
Read Through & Write Behind
Read-ThroughUser asks for data from the cacheIf the data is found in the cache, it’s returnedIf data is not found, the cache fetches the data using a specialized providerData is cached and returned to the user
Write-BehindUser asks to write/update data in the cacheThe cache stores the data in memoryUsing a specialized provider, the cache stores the data in permanent storage
Writing a provider is not hard, but not always trivialYou need to be able to map from the key to a “query” (database, web service, mainframe, etc.)
CacheProvide
r
CacheGuidance
41
When to use RT and WB
When it’s important to lower data round-tripsWhen clients always need non-stale data
Example: Inventory levelsOnly achievable if the data being manipulated is access exclusively through the cache
When wanting to save time and/or moneyExamples: Remote data source, complex queries, pay-per-query, data obtained from a MainframeAchievable by exposing the data source only through the cache
In most cases the RT/WB data will be dataResource
Implementing Caching
43
Windows Server AppFabric Cache
Cache runs on a cluster of serversCalles a “Cache Cluster”
All the servers in a cluster are part of a “fabric”Managed using PowerShell and behave as a single entityCommunicate, CooperateShare configuration via SQL or XML in a file share
Cache Cluster typically separate from the client (web) tierEach node runs the Cache as a Windows ServiceMore memory is better
Max recommended is 16GB per node
44
Windows Azure Shared Cache
On demand provisioning of cache resources128MB to 4GB per named cache
You don’t have to manage the cache clusterIncrease & decrease capacity using portal without losing data*
Compatible API with some differences: No H/A, notifications, disabling eviction, regions and tagsDefault expiration set at 48 hours (can change per-object)Connection is to a named cache, not cache cluster
45
Windows Azure Caching (Preview)
Full client API parity with Windows Server AFC
Option #1: Added to an existing cloud roleUses the instance’s memory (control over percentage used)Scales with roleNo additional cost
Option #2: Used a dedicated Worker RoleUses as much memory as you wantScales separately from existing rolesCosts like any other compute node
CacheGuidance
46
Choose the Right Cloud Caching Model
In all cases, it’ll be the new model No predefined cache sizes and much larger cachesFlexibility, control, and isolation of cache from other usersLower latencyNo extra cost on top of compute
47
What In-Role Dedicated SharedFeatures Full Full Limited
Cost No additional Pay for role Pay by memory
Scalability With app / tier Separate Separate
Competition for local resources
Yes No No
Memcached API compatible
Yes Yes No
Provisioning and Administration
Configuration Configuration Provision through portal
ASP.NET Caching Yes Yes Yes
Expiration Policy None, Absolute, Sliding Window (from last access)
None, Absolute, Sliding Window (from last access)
Absolute (48hr)
High Availability Yes Yes No
Notifications Yes Yes No
Disabling eviction Yes Yes No
Regions and Tags Yes Yes No
48
Caching Does Little to Reduce Cost
Accessing cloud data is not free, but it’s very cheapAzure storage access ($0.01 for 10K storage transactions)Cross data-center traffic ($0.12 per 1GB)
Cache access in same geo-region is freeConsider cost of cache
Windows Azure Shared CacheDedicated cache instances(No additional cost for the on-instance cache)
CacheGuidance
49
Use Cloud Caching for the Right Reasons
Speed of access to dataAzure SQL databaseAzure storageRemote storage and services
Avoiding hitting rate limitsUp to 5,000 entities/messages/blobs per second per accountUp to 500 messages per second from a single queueUp to 500 entities per second from a single table partition
CacheGuidance
50
Understand the Implications of Cloud Caching
Your cloud role resources are restricted
Co-located cache deploymentCompetes for local memoryCompetes for local CPUCompetes for the network
Virtual Machine
Size CPU Cores Memory
Allocated Bandwidth
(MBPS)
Temporary Local
Storage
Maximum Data Disks (1 TB each)
ExtraSmall Shared 768 MB 5 20 GB 1
Small 1 1.75 GB 100 50 GB 2
Medium 2 3.5 GB 200 100 GB 4
Large 4 7 GB 400 200 GB 8
Extra Large 8 14 GB 800 400 GB 16
CacheConcepts
51
ASP.NET Session State Provider
Stores session state in shared memoryRequired for Web FarmsFaster than SQL
Simple configuration change to your web app Enables session state to be cached across a clusterUses cache-aside, not RT+WB (session is Activity Data)TIP: Use HA to reduce the chances of sessions state loss
<sessionState mode="Custom" customProvider="SessionStoreProvider"> <providers> <add name="SessionStoreProvider“ type="Microsoft.ApplicationServer.Caching.DataCacheSessionStoreProvider, Microsoft.ApplicationServer.Caching.Client, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35“ cacheName=“MyCache"/> </providers></sessionState>
CacheConcepts
52
ASP.NET Page Output Caching
Cache full HTTP responses (page output caching)Plugs into the ASP.NET 4 output cache provider extensibility pointSimple configuration + page updates
<caching> <outputCache defaultProvider="DistributedCache"> <providers> <add name="DistributedCache" type="Microsoft.Web.DistributedCache.DistributedCacheOutputCacheProvider, Microsoft.Web.DistributedCache" cacheName="default" dataCacheClientName="default" /> </providers> </outputCache></caching>
<%@ OutputCache Duration="60" VaryByParam="*" %>
Summary
54
Summary
Caching is important to application performance and scalability, but it needs to be centralizedWindows Server AppFabric Cache is a free, scalable, distributed, in-memory cacheWindows Azure Caching (Preview) doesn’t cost extra, and gives all the benefits of the on-premises cacheUnderstanding the type of data you cache is key to your caching strategy and implementation choice
What: Reference Data, Resource Data, Activity DataHow: Cache Aside, RT/WB, Client Side Caching, Versions, Concurrency, High Availability, …
55
What Next?
Analyze & categorize your dataImplement caching correctly & efficiently
ASP.NET Session Provider is an easy startMeasure & Optimize usageMake things run FAST!
Related Content
AZR309 - Windows Azure Caching: New Capabilities in the Next Release
Resources
Connect. Share. Discuss.
http://northamerica.msteched.com
Learning
Microsoft Certification & Training Resources
www.microsoft.com/learning
TechNet
Resources for IT Professionals
http://microsoft.com/technet
Resources for Developers
http://microsoft.com/msdn
Complete an evaluation on CommNet and enter to win!
Please Complete an Evaluation Your feedback is important!
Multipleways to Evaluate Sessions
Scan the Tagto evaluate thissession now on myTechEd Mobile
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to
be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS
PRESENTATION.