hazelcast - in-memory datagrid
DESCRIPTION
Hazelcast is an easy to use but scalable in-memory datagrid and distributed executor framework. It enables you to build applications having a big requirement on memory or that needs to scale horizontally.TRANSCRIPT
HAZELCAST 3IN-MEMORY DATAGRID
www.hazelcast.com
WHO AM IChristoph Engelbert (@noctarius2k)8+ years of professional Java development5+ years of backend developmentSpecialized to performance, GC, traffic topicsWas working for int. companies as Ubisoft and HRSSince November 2013 official Hazelcast HackerApache DirectMemory / Lightning Committer and PMCDeveloper CastMapR - MapReduce on Hazelcast 3
www.hazelcast.com
HAZELCAST IN KEYWORDSIn-Memory DataGridDistributed CacheDistributed Execution FrameworkNoSQLClusteringScalabilityPartitioningCloud ReadyOpenSource - Apache License 2.0
www.hazelcast.com
WHO USES HAZELCAST?
and many more ...
www.hazelcast.com
USECASESScale your applicationDistribute and share dataPartition your dataDistribute messagesProcess in parallel on multiple machinesLoad balancing
www.hazelcast.com
ALTERNATIVES?Oracle CoherenceIBM eXtreme ScaleVMware GemfireGigaspacesRedhat InfinispanGridgainTerracotta
www.hazelcast.com
DIFFERENCES?Easy to useLicense / CostLightweightFeatures
www.hazelcast.com
FEATURESJava Collection API
Map, Queue, Set, ListMultiMapTopic (PubSub)Java Concurrency API
Lock, Semaphore, CountDownLatch, ExecutorServiceTransactionsCustom SerializationOff-Heap supportNative client: C#, C++, Java, REST, memcached
www.hazelcast.com
EASY API// Creating a new Hazelcast nodeHazelcastInstance hz = Hazelcast.newHazelcastInstance();
// Getting a Map, List, Topic, ...Map map = hz.getMap("MapName");List list = hz.getList("ListName");ITopic topic = hz.getTopic("TopicName");
// Shutting down the nodehz.shutdown();
www.hazelcast.com
HOW DOES IT WORK?
www.hazelcast.com
DATA PARTITIONING (1/2)Multiple partitions per nodeConsistent Hashing: hash(key) % partitioncountOption to control partitioning: "key@partitionkey"Possibility to find key owner for every keySupport for Near-Caching and executions on key ownerAutomatic Fault-ToleranceSynchronous and Asynchronous backupsDefine sync / async backup counts
www.hazelcast.com
With 4 cluster nodes every server holds1/4 real data and 1/4 of backups
DATA PARTITIONING (2/2)
www.hazelcast.com
A HAZELCAST NETWORK
www.hazelcast.com
HAZELCAST IN NUMBERSDefault partition amount 271Any partition amount possibleBiggest cluster 100+ membersHandles 100k+/sec messages using a topicMax datasize depends on RAMOff-Heap for low GC overhead
www.hazelcast.com
COMMUNITY VS. ENTERPRISEFeature Community EnterpriseJava Collection API X XJava Concurrency API X XSSL Socket X XElastic Memory (Off-Heap) XJAAS Security / Authentication XManagement Center X
www.hazelcast.com
CODE SAMPLES
www.hazelcast.com
EASY TO UNITTESTpublic class SomeTestCase {
private HazelcastInstance[] instances;
@Before public void before() throws Exception { // Multiple instances on the same JVM instances = new HazelcastInstance[2]; instances[0] = Hazelcast.newHazelcastInstance(); instances[1] = Hazelcast.newHazelcastInstance(); } @After public void after() throws Exception { Hazelcast.shutdownAll(); }}
www.hazelcast.com
SERIALIZATION// java.io.Serializablepublic class User implements Serializable {}
// or java.io.Externalizablepublic class User implements Externalizable {}
// or (com.hazelcast.nio.serialization).DataSerializablepublic class User implements DataSerializable {}
// or new in Hazelcast 3 (multi version support) Portablepublic class User implements Portable {}
www.hazelcast.com
MAPinterface com.hazelcast.core.IMap<K, V> extends java.util.Map, java.util.ConcurrentMap
HazelcastInstance hz = getHazelcastInstance();
IMap<String, User> hzMap = hz.getMap("users");hzMap.put("Peter", new User("Peter", "Veentjer"));
Map<String, User> map = hz.getMap("users");map.put("Peter", new User("Peter", "Veentjer"));
ConcurrentMap<String, User> concurrentMap = hz.getMap("users");concurrentMap.putIfAbsent("Peter", new User("Peter", "Veentjer"));
User peter = map.get("Peter");
www.hazelcast.com
LISTinterface com.hazelcast.core.IList<E> extends java.util.List
HazelcastInstance hz = getHazelcastInstance();
List<User> list = hz.getList("users");list.add(new User("Peter", "Veentjer"));
User peter = list.get(0);
www.hazelcast.com
QUEUEinterface com.hazelcast.core.IList<E> extends java.util.List
HazelcastInstance hz = getHazelcastInstance();
BlockingQueue<Task> queue = hz.getQueue("tasks");
queue.offer(newTask());queue.offer(newTask(), 500, TimeUnit.MILLISECONDS);
Task task = queue.poll();Task task = queue.poll(100, TimeUnit.MILLISECONDS);Task task = queue.take();
www.hazelcast.com
LOCK (1/3)interface com.hazelcast.core.ILock extends java.util.concurrent.locks.Lock
www.hazelcast.com
LOCK (2/3)HazelcastInstance hz = getHazelcastInstance();
// Distributed ReentrantLock lock = hz.getLock("myLock");lock.lock();try { // Do something} finally { lock.unlock();}
www.hazelcast.com
LOCK (3/3)HazelcastInstance hz = getHazelcastInstance();
// Map (Row-)locksIMap<String, User> map = hz.getMap("users");map.lock("Peter");try { // Do something with Peter} finally { map.unlock("Peter");}
www.hazelcast.com
TOPIC / PUBSUBpublic class Example implements MessageListener<String> { public void sendMessage { HazelcastInstance hz = getHazelcastInstance(); ITopic<String> topic = hz.getTopic("topic"); topic.addMessageListener(this); topic.publish("Hello World"); } @Override public void onMessage(Message<String> message) { System.out.println("Got message: " + message.getMessageObject()); }}
www.hazelcast.com
EXECUTORSERVICEpublic interface com.hazelcast.core.IExecutorService extends java.util.concurrent.ExecutorService
HazelcastInstance hz = getHazelcastInstance();
IExecutorService es = hz.getExecutorService("name");es.executeOnAllMembers(buildRunnable());es.executeOnKeyOwner(buildRunnable(), "Peter");es.execute(buildRunnable());
Map<..> futures = es.submitToAllMembers(buildCallable());Future<..> future = es.submitToKeyOwner(buildCallable(), "Peter");
es.submitToAllMembers(buildCallable(), buildCallback());es.submitToKeyOwner(buildCallable(), "Peter", buildCallback());
www.hazelcast.com
ADVANCED TECHNIQUES
www.hazelcast.com
ADVANCED TECHNIQUESIndexing keys, values and value propertiesDistributed SQL-like queryWrite-Behind / Write-Through persistenceRead-Through (if key not loaded use MapLoader)TransactionsEntryListeners / EntryProcessorsAutomatic evictionControl partitioning (Version 3.1)and many more ...
www.hazelcast.com
CODE SAMPLES
www.hazelcast.com
INDEXINGIMap<String, User> map = Hazelcast.getMap("users");map.addIndex("age", true); // orderedmap.addIndex("active", false); // not ordered
<map name="employees"> ... <indexes> <index ordered="true">age</index> <index ordered="false">name</index> </indexes></map>
www.hazelcast.com
DISTRIBUTED SQL-LIKE QUERIESIMap<String, User> map = Hazelcast.getMap("users");Predicate predicate = new SqlPredicate("active AND age <= 30");
Set<User> users = map.values(predicate);
Set<Entry<String, User>> entries = map.entrySet(predicate);
www.hazelcast.com
MAPLOADER / MAPSTOREpublic class MapStorage implements MapStore<String, User>, MapLoader<String, User> { // Some methods missing ... @Override public User load(String key) { return loadValueDB(key); } @Override public Set<String> loadAllKeys() { return loadKeysDB(); } @Override public void delete(String key) { deleteDB(key); } @Override public void store(String key, User value) { storeToDatabase(key, value); }}
<map name="users"> <map-store enabled="true"> <class-name>com.hazelcast.example.MapStorage</class-name> <write-delay-seconds>0</write-delay-seconds> </map-store></map>
www.hazelcast.com
TRANSACTION (1/2)HazelcastInstance hz = getHazelcastInstance();
final Map map = hz.getMap("default");final Queue queue = hz.getQueue("default"); hz.executeTransaction(new TransactionalTask<Void>() { @Override public Void execute(TransactionalTaskContext context) { Tweet tweet = (Tweet) queue.poll(); processTweet(tweet); map.put(buildKey(tweet), tweet); return null; }});
www.hazelcast.com
TRANSACTION (2/2)HazelcastInstance hz = getHazelcastInstance();
TransactionContext context = hz.newTransactionContext();context.beginTransaction();
TransactionalMap map = context.getMap("default");TransactionalQueue queue = context.getQueue("default");
try { Tweet tweet = (Tweet) queue.poll(); processTweet(tweet); map.put(buildKey(tweet), tweet); context.commitTransaction();
} catch (Exception e) { context.rollbackTransaction();}
www.hazelcast.com
Force location of corresponding data in the same partitionby providing a special partition key
CONTROL PARTITIONING
HazelcastInstance hz = getHazelcastInstance();
Map users = hz.getMap("users");users.put("Peter@Peter", new User("Peter", "Veentjer"));
Map friends = hz.getMap("friends");friends.put("Peter-Chris@Peter", new User("Christoph", "Engelbert"));friends.put("Peter-Fuad@Peter", new User("Fuad", "Malikov"));
www.hazelcast.com
WHAT ELSE?
www.hazelcast.com
SPI (NEW IN HAZELCAST 3)Possibility to build own distributed datastructuresHook into datastructure eventsImplement your own services (like RemoteInvocation,MapReduce)React on membership eventsManipulate migrations on your purposeHandle splitbrain eventsand many more ...
www.hazelcast.com
EXTERNAL ADDONShazelgrails - Hazelcast integration for Grailshazelmr / castmapr - MapReduce for Hazelcast 2.x / 3.xhazelblast - Remote Invocationhazelcast-actors - Actor frameworkmaybe your's next?
www.hazelcast.com
@noctarius2k@hazelcast
[email protected]://www.hazelcast.com
http://github.com/hazelcast/hazelcast
THANK YOU!ANY QUESTIONS?
Images: www.clipartist.info, Gnome Nebula Theme, KDE theme
www.hazelcast.com