appfabric velocity
DESCRIPTION
Presentation I did on AppFabric velocity while it was in beta, as in before it was called AppFabricTRANSCRIPT
Session Code: NE.13
Velocity, distributed caching
Dennis van der Stelt
Introduction
• Class-A– Kennisprovider– Microsoft development– Training / Coaching– www.class-a.nl
• Dennis van der Stelt– Trainer / Coach– Weblog at http://bloggingabout.net/blogs/dennis/
Agenda
• Challenges for webfarms• What is Velocity?• Options in caching in Velocity• Velocity features• Caching strategies• Best practices
Challenges for webfarms
Load balancer
Web server A
Web server B
Web server C
Solution : sticky sessions
Load balancer
Web server A
Web server B
Web server C
1. Uneven load balancing2. Lost sessions3. Single point of failure4. Need to drain state
Solution : Use DBMS
Load balancer
Web server A
Web server B
Web server C
1. Doesn’t scale2. Creates a performance
bottleneck3. Clustering is costly and fault
sensitive
What is Velocity?
What is Velocity?• An explicit, distributed, in-memory application
cache for all kinds of data (CLR objects, XML, Binary data, etc.)– Flows "memory" across machines into a unified
cache
Unified Cache View
Clients can be spread across machines or
processes
Clients Access the Cache as if it was a large single cache
Cache Layer distributes data
across the various cache nodes
Velocity in a webfarm
Load balancer
Web server A
Web server B
Web server C
Why Velocity?
Operation Throughput
Latency
Read 2k 30,000 / sec
3 – 4 ms
Write 2k 18,000 / sec
3 ms
Operation Servers
Throughput
Read 2k 1 30,000 / sec
Read 2k 2 58, 600 / sec
Read 2k 3 85, 500 / sec
Velocity
* Share data across applications No more sticky routing
* Peformance
* Scale out
MSDN Forums5 Velocity Servers, 40GB Cache
98% of all calls to database come from cacheResult is from 60% load on SQL Server, down to 2%
Key features of Velocity
• Caches any serializable CLR object• Enterprise scale
– Up to hundreds of computers– Dynamic scaling
• Runs as a service• Automatic load balancing
Key features of Velocity
• Tight integration with ASP.NET– Use as session store– Cache application data
• Integration with administration and monitoring tools as System Center, etc.
• Cache-aside architecture for version 1• Support for multiple client languages
– PHP, C#, C++, etc.
Velocity installation
• Needed for installation– .NET Framework 3.5– Windows XP/Vista/Server (2003/2008)
• Firewall exceptions on port 22233-22235• Configuration
– SQL Server– Network share
single-point-of-failure?10 concurrent connections?
Basic terminology in Velocity
• Cache host• Cache cluster• Cluster configuration storage• Named cache• Region• Cache item• Tags
Machine 1Cache host A
Cache host B
Machine 2Cache host C
Machine 2Cache host DNamed cache : Product catalog
Named cache : ShoppingCart
Region A
Working with Velocity
// Create instance of CacheFactory, which reads app.config DataCacheFactory factory = new DataCacheFactory();
// Get a named cache from the factory DataCache cache = factory.GetCache("default");
// Cache.Put(string key, object value)cache.Add("SDC", new SDNConference()); // Cache.Get(string key);var meeting = (SDNConference)cache.Get("SDC");
// Via indexers is also an optioncache["what"] = new Object();Object o = cache["what"];
Types of cache
•Data partitioned across all nodes
•Used for scale & availability
Partitioned
•Data replicated across all nodes
•Used for high availability
•All reads occur on primary host
Replicated
•Payload stays in object form•No
serialization
•No extra network hops
Local
Partitioned cache
Application 2Application 1
Cache2Cache1
Primary RegionsPrimary Regions
T1, “A”
Cache3
Primary Regions
T3, “A”
Routing table Routing table
Put(“T5”, “D”)
T5, “D”
Routing table Routing table Routing table
33% 33% 33%
Replicated cache (high availability)
Application 2Application 1
Cache2Cache1
Primary RegionsPrimary Regions
T1, “A”
Cache3
Primary Regions
T3, “A”
Routing table Routing table
Put(“T5”, “D”)
Secondary Regions Secondary Regions Secondary Regions
T5, “D”
T5, “D” T3, “A” T1, “A”
Get(“T5”)
Routing table Routing table Routing table
Cache clients
Simple client
• Has to request where objects live
Routing Client
• Has knowledge of where objects live
• Keeps in sync with hosts
• Performance benefits
• Extra network communication
Local cache
• No pre-installed host needed
• Your application is the host
• Configurable• Stored in
deserialized state!
Partitioned cache fronted by local cache
Velocity Client2Velocity Client1
Cache2Cache1
Primary RegionsPrimary Regions
T1, “A”
Cache3
Primary Regions
T3, “A”
Put(T5, “D”)
Get(T5)
T5, “D”
Local cache
Routing layer
Local cache
Routing layer
T5, “D” T5, “D”
Velocity features
Regions
// Without regionscache.Put("MyKey", sdnConference); // With regionscache.CreateRegion("MyRegion", true);cache.Put("MyKey", sdnConference, "MyRegion"); var result = (SDNConference)cache.Get("MyKey", "MyRegion");
• Regions don’t distribute• But will replicate when high availability is on!
Tags
var starWarsTag = new DataCacheTag("StarWars");
var tags = new List<DataCacheTag>();tags.Add(starWarsTag);tags.Add(new DataCacheTag("Force"));tags.Add(new DataCacheTag("Sith")); cache.Add("MyKey", "A New Hope", tags, "StarWarsRegion"); var result = cache.GetObjectsByTag(starWarsTag, "StarWarsRegion"); foreach (var item in result){ Console.WriteLine("{0} has value of {1}", item.Key, item.Value);}
Optimistic locking
DataCacheItemVersion versionWillChange;DataCacheItemVersion versionWithError; // First get the current version 2 timescache.Get("MyKey", out versionWillChange);cache.Get("MyKey", out versionWithError); // We change the key, version hasn't changed in Velocity yet.cache.Put("MyKey", "MyNewValue", versionWillChange); // Version has changed with previous update, this will #failcache.Put("MyKey", "MyErrorValue", versionWithError);
Pessimistic locking
DataCacheLockHandle lockHandle = null;DataCacheLockHandle secondLockHandle = null; // Lock our objectcache.GetAndLock("MyKey", new TimeSpan(0, 0, 10), out lockHandle); // This will still workstring result = (string)cache.Get("MyKey"); // Try to lock for 2nd time -> #failcache.GetAndLock("MyKey", new TimeSpan(0, 0, 10), out secondLockHandle); // This will break the lock!!!cache.Put("MyKey", "MyNewValue");
Notification callback
DataCacheOperation filter = DataCacheOperation.AddItem;cache.AddItemLevelCallback("MyKey", filter, callback);cache.AddRegionLevelCallback("Region", filter, callback);cache.AddFailureNotificationCallback(failCallback); cache.Add("MyKey", "MyInitialValue");
public static void callback (string myCacheName, string myRegion, string myKey, DataCacheItemVersion itemVersion, DataCacheOperation OperationId, DataCacheNotificationDescriptor nd){ //display some of the delegate parameters Console.WriteLine("Region : " + myRegion); Console.WriteLine("Key : " + myKey);}
Cluster configuration
<dataCacheClient deployment="routing"> <localCache isEnabled="true" sync="TTLBased" objectCount="100000" ttlValue="300" /> <clientNotification pollInterval="300" /> <!-- cache host(s) --> <hosts> <host name="localhost" cachePort="22233" cacheHostName="DistributedCacheService"/> </hosts></dataCacheClient>
ASP.NET Session integration
• SessionStoreProvider class– Plugs into ASP.NET Session store– Stores session state in Velocity
• Scale– Session information available at all ASP.NET
Nodes• High availability
– Session data is backed up on addditional machines– Resilient to machine or process failures
Performance
Achieving scale and performance
• Scale on data size– More machines => more memory to cache
• Scale on cache throughput– Throughput : Operations/sec from entire cache– More machines : keys distributed across more
machines => better throughput• Performance
– High performance by scaling out data and processing– Increased memory– Increased processing
• Workload distributed across multiple cache nodes
Achieving scale and performance
Single ServerThroughput Increases with Increasing Load
Until Server Saturation
Load
Throughput
Latency
Server 2 AddedThroughput Increases
Latency DecreasesUntil Server Saturation
Server 3 AddedThroughput Increases
Latency Decreases
Caching strategies
Velocity
• Is Velocity next-next-finish?– Think about the consequences!– From in-proc. cache to out-of-process cache– Multiple network cards for higher bandwith
• Don’t disturb your normal web visitors• Velocity can bring network card to its knees
– Computers are communicating data• Can you transfer lots of data, allowed to do so?• What about security?
Best practices
• Develop a caching strategy– When and what to cache– Selection of cache keys
• Create guidelines for caching– Make sure developers live by the strategy– Caching is hard to do right
• Code once, measure twice
Best practices
• Cache close to where you need the items– Usually presentation or service layer
• Cache wisely– Think about cache item lifetimes, eviction– Don’t cache to get performance, but to
improve performance• Define a key naming convention
– Avoid conflicts of cache entries
Velocity and the future
• Available mid 2009 (huh?)• Velocity is cache provider in .NET 4.0• Version 2
– Will be available in the cloud– LINQ over Velocity– Output caching provider– Cache through
• Read through & Write behind
– Grid computing
Review
• Distributed cache is the future for performance demanding enterprises– Linkedin, Slashdot, Facebook, Flickr,
Wikipedia• Velocity makes this possible• A giant hashtable with performance,
scalability and failover.
Thank you for your attention
• Dennis van der Stelt– [email protected]– http://twitter.com/dvdstelt/– http://bloggingabout.net/blogs/dennis/
Resources to more information can be found here, incl. this slidedeck.
Evaluation form
Vul je evaluatieformulier in en maak kans op een van de prachtige prijzen!!
Fill out your evaluation form and win one of the great prizes!!
Session Code: NE.13