implementing high availability caching with memcached
DESCRIPTION
Typical Memcached deployments do not comprehensively address web site requirements for high availability. Depending on your web architecture, a single failure can disable your web caches. This presentation offers real world solutions to solving high availability challenges common to large, dynamic websites with Memcached, specifically: * Options and benefits for deploying high availability services within Memcached * How companies are approaching high availability * Considerations on building and deploying high availability o Recommendations for a typical Memcached environment o Open source tools available o High level costs for deploymentTRANSCRIPT
![Page 1: Implementing High Availability Caching with Memcached](https://reader031.vdocument.in/reader031/viewer/2022020207/54b799994a79591d4a8b47c4/html5/thumbnails/1.jpg)
Creating High Availability Services for YourMemcached Environment
25 August 2009
Presented by:
Bill Takacs – Director, Product Management
![Page 2: Implementing High Availability Caching with Memcached](https://reader031.vdocument.in/reader031/viewer/2022020207/54b799994a79591d4a8b47c4/html5/thumbnails/2.jpg)
2 : Copyright 2009 Gear6 Inc.
Agenda
• Overview of Memcached
• High Availability defined
• Why HA Memcached
• Options for HA Memcached
• Gear6 Solution
![Page 3: Implementing High Availability Caching with Memcached](https://reader031.vdocument.in/reader031/viewer/2022020207/54b799994a79591d4a8b47c4/html5/thumbnails/3.jpg)
3 : Copyright 2009 Gear6 Inc.
Memcached
“A high performance, distributed memory objectcaching system, generic in nature, but intended foruse in speeding up dynamic web applications by
alleviating database load”Ref: http://www.danga.com/memcached/
• Big hash table
• Created by Danga Interactive for Live Journal
• Significantly reduced database load
• Perfect for web sites with high database load
• In use by Facebook, Twitter, MyYearBook, others
![Page 4: Implementing High Availability Caching with Memcached](https://reader031.vdocument.in/reader031/viewer/2022020207/54b799994a79591d4a8b47c4/html5/thumbnails/4.jpg)
4 : Copyright 2009 Gear6 Inc.
More on Memcached
• Takes advantage of available DRAM
• Open source
• Distributed under BSD license
• Server - Current version is 1.4» http://www.danga.com/memcached/download.bml
• Many Clients» http://code.google.com/p/memcached/wiki/Clients
![Page 5: Implementing High Availability Caching with Memcached](https://reader031.vdocument.in/reader031/viewer/2022020207/54b799994a79591d4a8b47c4/html5/thumbnails/5.jpg)
5 : Copyright 2009 Gear6 Inc.
How Does Memcached Work?
• Client - Server
• Hash Function
• Memcached» Two stage» Key / value pair
![Page 6: Implementing High Availability Caching with Memcached](https://reader031.vdocument.in/reader031/viewer/2022020207/54b799994a79591d4a8b47c4/html5/thumbnails/6.jpg)
6 : Copyright 2009 Gear6 Inc.
Memcached Server
• Slab allocator
• Libevent based
• Simple Protocol (no xml)
• Server has internal hash table
• Servers are dumb – they don’t know about each other
![Page 7: Implementing High Availability Caching with Memcached](https://reader031.vdocument.in/reader031/viewer/2022020207/54b799994a79591d4a8b47c4/html5/thumbnails/7.jpg)
7 : Copyright 2009 Gear6 Inc.
Memcached Server
• Limits» Key size = (250 bytes)» 1MB Limit» 32bit/64bit (maximum size of process)
• LRU» Least recently accessed items are cycled out» One LRU exists per “slab class”» LRU “evictions” need not be common
• Threads? Yes
![Page 8: Implementing High Availability Caching with Memcached](https://reader031.vdocument.in/reader031/viewer/2022020207/54b799994a79591d4a8b47c4/html5/thumbnails/8.jpg)
8 : Copyright 2009 Gear6 Inc.
Memcached Clients
• Client hashes key to server list
• Serializes the object
• Many client libraries
• Authentication
![Page 9: Implementing High Availability Caching with Memcached](https://reader031.vdocument.in/reader031/viewer/2022020207/54b799994a79591d4a8b47c4/html5/thumbnails/9.jpg)
9 : Copyright 2009 Gear6 Inc.
Commands
• Storage commands:» Set / Add / Replace /
Append / Prepend / CAS
• Retrieval commands:» Get / Gets
• Delete / increment /decrement
• Stats
![Page 10: Implementing High Availability Caching with Memcached](https://reader031.vdocument.in/reader031/viewer/2022020207/54b799994a79591d4a8b47c4/html5/thumbnails/10.jpg)
10 : Copyright 2009 Gear6 Inc.
High Availability
• High availability: Systemdesign and implementationthat ensures a certainabsolute degree ofoperational continuityduring a given measurementperiod.
• Organizations» OpenSAF Foundation» SAFourm
• HA for Memcache
![Page 11: Implementing High Availability Caching with Memcached](https://reader031.vdocument.in/reader031/viewer/2022020207/54b799994a79591d4a8b47c4/html5/thumbnails/11.jpg)
11 : Copyright 2009 Gear6 Inc.
Key Concepts
• Clustering
• Failover
• Replication
![Page 12: Implementing High Availability Caching with Memcached](https://reader031.vdocument.in/reader031/viewer/2022020207/54b799994a79591d4a8b47c4/html5/thumbnails/12.jpg)
12 : Copyright 2009 Gear6 Inc.
The case for HA Memcache
• What Memcache is NOT:» A persistent data store» A database» Application specific» Clustered» A large object cache
• The intent is if its not in cache hit the DB
• Great when your site’s traffic is small
• Let’s look at the evolution of a site
![Page 13: Implementing High Availability Caching with Memcached](https://reader031.vdocument.in/reader031/viewer/2022020207/54b799994a79591d4a8b47c4/html5/thumbnails/13.jpg)
13 : Copyright 2009 Gear6 Inc.
The Beginning
Web Servers
MySQLDatabase
![Page 14: Implementing High Availability Caching with Memcached](https://reader031.vdocument.in/reader031/viewer/2022020207/54b799994a79591d4a8b47c4/html5/thumbnails/14.jpg)
14 : Copyright 2009 Gear6 Inc.
Dynamic content:
Result: Site slows down or worse site unavailable
Web traffic grows
Web Servers
MySQLDatabase
MemcachedMemcachedMemcachedMemcached
MemcachedMemcached
![Page 15: Implementing High Availability Caching with Memcached](https://reader031.vdocument.in/reader031/viewer/2022020207/54b799994a79591d4a8b47c4/html5/thumbnails/15.jpg)
15 : Copyright 2009 Gear6 Inc.
MemcachedMemcached
Increases performance but as the site grows so does Memcached
MemcachedMemcached
Add Memcached as traffic increases
Web Servers
MySQLDatabase
![Page 16: Implementing High Availability Caching with Memcached](https://reader031.vdocument.in/reader031/viewer/2022020207/54b799994a79591d4a8b47c4/html5/thumbnails/16.jpg)
16 : Copyright 2009 Gear6 Inc.
MemcachedMemcached
MemcachedMemcached
Even more traffic
Web Servers
MySQLDatabases
Memcache Server(s)
MemcachedMemcached
Reliance on memcached for performance
![Page 17: Implementing High Availability Caching with Memcached](https://reader031.vdocument.in/reader031/viewer/2022020207/54b799994a79591d4a8b47c4/html5/thumbnails/17.jpg)
17 : Copyright 2009 Gear6 Inc.
Options for HA Memcached
Not Many…
• DIY
• Or - Find a solution that leverages memcached thathas high availability features
![Page 18: Implementing High Availability Caching with Memcached](https://reader031.vdocument.in/reader031/viewer/2022020207/54b799994a79591d4a8b47c4/html5/thumbnails/18.jpg)
18 : Copyright 2009 Gear6 Inc.
Consistent Hashing
• Why?
• How does it work?
• Where
![Page 19: Implementing High Availability Caching with Memcached](https://reader031.vdocument.in/reader031/viewer/2022020207/54b799994a79591d4a8b47c4/html5/thumbnails/19.jpg)
19 : Copyright 2009 Gear6 Inc.
Consistent Hashing - Example
• PHP – Flexihash – code.google.com/p/flexihash<?php$hash = new Flexihash();// bulk add$hash->addTargets(array('cache-1', 'cache-2', 'cache-3'));
// simple lookup$hash->lookup('object-a'); // "cache-1"$hash->lookup('object-b'); // "cache-2"
// add and remove$hash ->addTarget('cache-4') ->removeTarget('cache-1');
// lookup with next-best fallback (for redundant writes)$hash->lookupList('object', 2); // ["cache-2", "cache-4"]
// remove cache-2, expect object to hash to cache-4$hash->removeTarget('cache-2');$hash->lookup('object'); // "cache-4"
![Page 20: Implementing High Availability Caching with Memcached](https://reader031.vdocument.in/reader031/viewer/2022020207/54b799994a79591d4a8b47c4/html5/thumbnails/20.jpg)
20 : Copyright 2009 Gear6 Inc.
Replication
• How» Repcache» Libmemcached
• Why» Two datacenters – master / slave dc’s» Keys stored on multiple machines to make them ‘more”
available
![Page 21: Implementing High Availability Caching with Memcached](https://reader031.vdocument.in/reader031/viewer/2022020207/54b799994a79591d4a8b47c4/html5/thumbnails/21.jpg)
21 : Copyright 2009 Gear6 Inc.
Replication - Repcache Example
• Downloaded the memcached-(version)-repcached-(version).tar.gz source and simply compiled it.
./configure --enable-replicationmakesudo make install
Start two instances of memcached on ports 11211 & 11222
btak@csn07:~/php$ memcached -p 11211 -m 64 -x 127.0.0.1 -vreplication: connect (peer=127.0.0.1:11212)replication: marugoto copyingreplication: closereplication: listen
btak@csn07:~/php$ memcached -p 11222 -m 64 -x 127.0.0.1 -vreplication: connect (peer=127.0.0.1:11212)replication: marugoto copyingReplication: start
![Page 22: Implementing High Availability Caching with Memcached](https://reader031.vdocument.in/reader031/viewer/2022020207/54b799994a79591d4a8b47c4/html5/thumbnails/22.jpg)
22 : Copyright 2009 Gear6 Inc.
Replication – Repcache Example Cont
Now set and get a value on instance onport 11211
btak@csn07:~$ telnet localhost 11211Trying 127.0.0.1...Connected to localhost.Escape character is '^]'.set hello 0 0 5worldSTOREDget helloVALUE hello 0 5worldEND
Connect to port 11222 and try gettingthis value
btak@csn07:~$ telnet localhost 11222• Trying 127.0.0.1...• Connected to localhost.• Escape character is '^]'.• get hello• VALUE hello 0 5• world• END
![Page 23: Implementing High Availability Caching with Memcached](https://reader031.vdocument.in/reader031/viewer/2022020207/54b799994a79591d4a8b47c4/html5/thumbnails/23.jpg)
23 : Copyright 2009 Gear6 Inc.
Replication – Repcache Example Cont
Suppose server on 11211 goes down
On port 11211Killedbtak@csn07:~/php$ memcached -11211
-m 64 -x 127.0.0.1 -vreplication: connect
(peer=127.0.0.1:11222)replication: marugoto copyingreplication: start
On port 11222<---snipreplication: closereplication: listenreplication: acceptreplication: marugoto startreplication: marugoto 2replication: marugoto owari<---snip-->
Let’s see if its replicated on 11211
btak@csn07:~$ telnet localhost 11211Trying 127.0.0.1...Connected to localhost.Escape character is '^]'.get helloVALUE hello 0 5worldENDget keyVALUE key 0 5myvalEND
![Page 24: Implementing High Availability Caching with Memcached](https://reader031.vdocument.in/reader031/viewer/2022020207/54b799994a79591d4a8b47c4/html5/thumbnails/24.jpg)
24 : Copyright 2009 Gear6 Inc.
Clustering
• www.linux-ha.org
• Provides high availability clustering solution
• Can enable memcached cluster
• Combine with consistent hashing
![Page 25: Implementing High Availability Caching with Memcached](https://reader031.vdocument.in/reader031/viewer/2022020207/54b799994a79591d4a8b47c4/html5/thumbnails/25.jpg)
25 : Copyright 2009 Gear6 Inc.
About Gear6
• First and leading providerof Memcached solutions
• Memcached solution including
• High density
• High Availability
• Advanced memorymanagement
• Enhanced reportingcapabilities
• Support for multi-tenancy
• Disruption freesoftware upgrades
• 100% client compatible
![Page 26: Implementing High Availability Caching with Memcached](https://reader031.vdocument.in/reader031/viewer/2022020207/54b799994a79591d4a8b47c4/html5/thumbnails/26.jpg)
26 : Copyright 2009 Gear6 Inc.
References
• http://dev.mysql.com/doc/mysql-ha-scalability/en/ha-memcached-using-deployment.html
• http://dev.mysql.com/doc/mysql-ha-scalability/en/ha-memcached-using-hashtypes.html
• http://jayant7k.blogspot.com/2009/04/memcached-replication.html
• http://www.lexemetech.com/2007/11/consistent-hashing.html
• http://www8.org/w8-papers/2a-webserver/caching/paper2.htmlhttp://www.last.fm/user/RJ/journal/2007/04/10/rz_libketama_-_a_consistent_hashing_algo_for_memcache_clients
• http://bazaar.launchpad.net/~libmemcached-developers/libmemcached/trunk/revision/539
![Page 27: Implementing High Availability Caching with Memcached](https://reader031.vdocument.in/reader031/viewer/2022020207/54b799994a79591d4a8b47c4/html5/thumbnails/27.jpg)
27 : Copyright 2009 Gear6 Inc.
Questions?
Thank you for attending“Creating High Availability Services for Your Memcached
Environment presented” by Gear6
Bill TakacsBill TakacsGear6Gear6
[email protected]@gear6.com+1 650 587 7118+1 650 587 7118