scalable resilient web services in .net
Post on 25-Dec-2014
689 Views
Preview:
DESCRIPTION
TRANSCRIPT
Designing and Delivering Scalable and Resilient Web Services
Ron JacobsSr. Technical Evangelist, Microsofthttp://blogs.msdn.com/rjacobs
Agenda
The Problem
A Possible Solution
New Possibilities
What to do next
Simple
Do the simplest thing that will possibly work
Scalability
Able to support the required quality of service as the system load increases
-Wiktionary
Typical Web Architecture
• Need to get routed to same machine (i.e. sticky sessions)Users
• Each machine round trips for data
• Data that is cached is stored in the memory of one server
Web Tier
• CPU and disk can get saturated due to traffic
• Service access is slow
Data Tier
Web Explosion
Database
IIS/ASP.NET Application
Web Site’s too
slow!!
Database is hot!!
Where did my
shopping cart go?
IIS/ASP.NET Application
IIS/ASP.NET Application
Servers are crashing
Services are slow
Agenda
The Problem
A Possible Solution
New Possibilities
What to do next
Data Near Processing
ASP.NET
Web ServiceDatabase
Smart Client Browser
Cache
CacheCache
CacheCache
Good but…
• Cache is scoped to machine / process– Machines die– Processes recycle
• Cache memory is limited
What if?
• You could have as much cache as you wanted?• You could share a giant cache across servers,
services and even clients?
• What if this was something you could simply add to the platform for 1free?
1Some features may require certain editions of Windows Server
Windows Server AppFabric
AppFabricCACHING
MONITORING
WORKFLOW HOSTING
SERVICE HOSTING
SCALE OUTHIGH AVAILABILITY MANAGEMENT
AppFabric Cache – Formerly known as Code Name “Velocity”
Unified Cache View
What is AppFabric Caching?
• An explicit, distributed, in-memory application cache for all kinds of data
Caching clients can be across machines or
processes
Clients Access the Cache as if it
was a large single cache
Cache Layer distributes data
across the various cache
nodes
AppFabric Cache
• No need for sticky sessionsUsers
• Cached data available to all serversWeb
Tier
• Multiple machines means scale and high-availability
• Lots of cache memory
Caching Tier
• Reduces load on database• Duplicate Service Calls
eliminated
Data Tier
Data Distribution - Partitioned Cache
… Web Tier
Cache TierCa
che
Ser
vice
Cach
e S
ervi
ce
Cach
e S
ervi
ce
A C EDB G IHF
Scale on Data Size - More machines => More memory to cache
Scale on Cache Throughput - More machines => keys distributed across
more machines => better throughput
ASP.Net App
Caching Client
ASP.Net App
Caching Client
ASP.Net App
Caching Client
AC ED BGI HF
Scale Test Output
Load
Throughput
Latency
1 Cache ServerAs load increases, throughput fails
to scale latency increases
Caching Tier
Add Second Cache Server
ThroughputLoad Max
Throughput increasesLatency
decreases
Caching Tier
Load
Latency
Add Third Cache ServerLoad
Throughput
Latency
Caching Tier
Associated Press
• Caches metadata and news
• Serves 16 million hits per day
• Increased the amount of cached data 6 times.
System.Web.Cache
AppFabric DataCache
Deployment
Cach
ing
Ser
vice
Cach
ing
Serv
ice
Cach
ing
Ser
vice
Configuration Store
Install AppFabric
Copy Client DLLs
… Application Application Application
<hosts> <host name="BL1CDB8083714“ cachePort="22233" cacheHostName="DistributedCacheService"/> ….. </hosts> <localCache isEnabled=“true" ../><security … />
Caching Access Layer Caching Access Layer Caching Access Layer.NET 3.5 SP1 OR .NET 4
Configure AppFabric
.NET 4
Update Web.config
Usage Pattern – Cache Aside (Explicit Caching)
// Read from CacheToy toyObj = (Toy) catalog.Get("toy-101");
Application
Cach
ing
Serv
ice
Database
// If Not present in the cacheif (toyObj == null){ // Read from backend.. toyObj = ReadFromDatabase(); // Populate Cache catalog.Put("toy-101", toyObj);
return toyObj;}
Caching Access Layer
Administration
• PowerShell cmdlets are used to administer the cache cluster
• Rich set of cmdlets for – Cache cluster management– Cache creation and monitoring
Hello AppFabric Cache
Demo
Using PowerShell
Remember – PowerShell can also be called from .NET Code!
AppFabric Cache Codeplex Tool
http://mdcadmintool.codeplex.com/
Demo
Security
• Domain Based Security Option– Domain Account / Local Account based Authentication– Only authorized servers can join the cluster– Only authorized clients can connect to the cluster
• Transport Level Security– Turn on/off Signing or Encryption
• Can turn off Cache Security– Use Firewalls, IPSec, VLANs to protect cache
grant-cacheallowedclientaccount RedDomain\Machine1$ grant-cacheallowedclientaccount RedDomain\John
AppFabric Caching Logical Hierarchy
• Host– Physical processes hosting AppFabric
Caching instance.• Named Caches
– Can span across machines– Defined in the configuration file
• Cache Item– Key, Payload (Object ), Tags, TTL,
Timestamps, Version• Regions
– Physically co-located Container of Cache Items
– May be implicit or explicitly created
Regions Region A
Key Payload Tags Key Payload Tags 121 xxxx “Toy” “Child”
123 yyyy “Toy” “Chair”..
Machine -> Cache Host -> Named Caches -> Regions -> Cache Items -> Objects
AppFabric Caching Service
Named Cache : Product Catalog
Named Cache : Electronics Inventory
AppFabric Caching Service
AppFabric Caching Service
AppFabric Caching Service
AppFabric Caching API// Create instance of cachefactory (reads appconfig)DataCacheFactory fac = new DataCacheFactory();
// Get a named cache from the factoryDataCache catalog = fac.GetCache("catalogcache");
// Simple Get/Putcatalog.Put("toy-101", new Toy("Puzzle", .,.));
// From the same or a different clientToy toyObj = (Toy)catalog.Get("toy-101");
// Region based Get/Putcatalog.CreateRegion("toyRegion");
// Both toy and toyparts are put in the same region catalog.Put("toy-101", new Toy( .,.), “toyRegion”);Catalog.Put("toypart-100", new ToyParts(…), “toyRegion”);
Toy toyObj = (Toy)catalog.Get("toy-101“,"toyRegion");
Access APIs – Tagging Items
Tag hotItem = new Tag("hotItem");
catalog.Put("toy-101", new Toy("Puzzle"), new Tag[]{hotItem}, “toyRegion”);
catalog.Put("toy-102", new Toy("Bridge"), “toyRegion”);
// From the same or a different clientList<KeyValuePair<string, object>> toys = catalog.GetAnyMatchingTag("toyRegion", hotItem);
Types of DataReference Activity Resource
Primary Read Only Read-Write Not shared Read-Write, Shared
Catalog Data Shopping Cart Auction Data/Seat Assignment
Web Tier
Distributed Cache
Shopping Cart
Grocery Catalog
Grocery Inventory
Grocery Shop
ApplicationApplication
AppFabric Caching Client
Reference Data – Performance• Catalog data doesn’t change often• Unnecessary network cost to access from different machines• Solution – Local Cache
Put(K2, v3)
Routing Table
Cache2Cache1
Primary for K2,V2
K2, V2
Primary for K1,V1
K1, V1
Cache3
Primary for K3,V3
K3, V3
AppFabric Caching ClientLocal Cache
Routing TableK2, V2
Get(K2)Get(K2)
K2, V3
Reference Data – Bulk Get
• Bulk Fetch from region– 200-300k ops per second– Fewer network calls
Catalog.BulkGet( new List<string>(){“toy-101”, “toy-
102”} , “toyRegion”);
Activity Data – Session Integration
…Session State
stored in AppFabric Caching
Cach
e S
ervi
ce
Cach
ing
Ser
vice
Load Balance RequestsNo more sticky routing
Cach
ing
Ser
vice
Scale your Session StoreDynamically
Highly Available
Drop in AppFabric Caching
SessionStoreProvider
Allows session state to be shared amongst multiple applications
Application
Caching Access Layer
Application ApplicationCaching Access Layer Caching Access Layer
<sessionState mode="Custom“ customProvider="SessionStoreProvider"><providers> <add name="SessionStoreProvider" type=“Microsoft.Data.Caching.DataCacheSessionStoreProvider, ClientLibrary“ cacheName="<YourNamedCache>"/></providers></sessionState>
ApplicationApplication
(K2, V2)
Cache2Cache1 Cache3
Primary for
Activity Data - Availability
Get(K2)
Primary for Primary for
K3, V3
AppFabric Caching ClientRouting Table
K2, V2
PUT
Secondary for
K2, V2
K1, V1
Secondary for
K3, V3
Secondary for
K1, V1
AppFabric Caching Client
Routing Table
K2, V2
Replication AgentK2, V2
Resource Data - Optimistic Locking
• GetCacheItem returns a version object• Every update to an object internally increments it's version• Supply the version obtained along with the Put/Remove• Put/Remove will succeed only if the passed in version matches the version
in the cache
Version Based Update
Time Client1 Client2 (Different Thread or process)
T0 CacheItem item = catalog.GetCacheItem(“PlayerRegion”, ”Zune”);
CacheItem item = catalog.GetCacheItem(“PlayerRegion”, ”Zune”);
T1 ((ZuneObject)item.Object).inventory --; ((ZuneObject)item.Object).inventory--;
T2 catalog.Put(“PlayerRegion”, “Zune”, item.Object, item.Version);
T3 catalog.Put(“PlayerRegion”, “Zune”, item.Object, item.Version);// Version mismatch// Client must retry again
Two clients access the same item
Both update the item
Second Client gets in first; put succeeds because item version matches; atomically
increments the version
First client tries put;Fails because the versions
don’t match
K1
Resource Data - Pessimistic Locking
• Take locks on non-existent keys– Allows you to co-ordinate calls for data
Client1: GetAndLock ("k1")
Client2: GetAndLock ("k1")
Client3: Get ("k1")
Regular Get succeeds
GetAndLock gets lock handle
Other GetAndLock on same item fails
Data Race
GET GETGET
Lock Non-Existent Key
GET/LOCK GET/LOCKGET/LOCKCa
che
Ser
vice
Cach
ing
Ser
vice
Cach
e S
ervi
ce
Composite Race
CALL WAITWAITCa
che
Ser
vice
Cach
ing
Ser
vice
Cach
e S
ervi
ce
Composite Race
PUTUNLOCK GETGET
Cach
e S
ervi
ce
Cach
ing
Ser
vice
Cach
e S
ervi
ce
Resource/Activity Data – Tracking Changes
• Cache Event notifications• Register on any client to notify changes• Batched Notifications
DataCache.RegisterCacheLevelCallback( int filter, DataCacheChangeCallback delegate);
DataCache.RegisterRegionLevelCallback( String region, int filter, DataCacheChangeCallback delegate);
DataCache.RegisterKeyLevelCallback( String region, String key, int filter, DataCacheChangeCallback delegate);
Application
Cache2Cache1
Primary for
K2, V2
Primary for
K1, V1
Cache3
Primary for
K3, V3
Scalable Notifications
AppFabric Caching Client
Routing Table
Register Notification for Key “K3"
Map Keys to Partition
Poll Required Nodes
Nodes Return List of Changes LSN Order
Partition: P2
Last LSN: 19
Call DelegateStore Last LSN
Change LogPartition P11 Add K22 Del
K32
Change Log
(Partition
P2)
18Del K32
19Del K43
Change Log33 Add K134 Del
K22
Agenda
The Problem
A Possible Solution
New Possibilities
What to do next
Data Center
Pre-Fetch
Hospital
Data Center
Pre-Fetch
HospitalRemote Clinic
Slow!!
WAN
Data Center
Pre-Fetch
HospitalRemote Clinic
Cach
e S
ervi
ce
WAN
Agenda
The Problem
A Possible Solution
New Possibilities
What to do next
Web Platform Installer
Select Enterprise
Install AppFabric
Install AppFabric
endpoint.tv
© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions,
it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
top related