infinispan in 50 minutes - jboss...•5.2 - february 2013 •non-blocking state transfer...

Infinispan in 50 minutesSanne Grinovero

Sanne Grinovero•Senior Software Engineer at Red Hat•Hibernate team

• lead of Hibernate Search• Hibernate OGM

•Infinispan• Search, Query and Lucene integrations

•Apache Lucene•JGroups

Who’s this guy?

Agenda•What is Infinispan•API•Key features•Three use cases•The path ahead

Infinispanl Open source highly scalable data grid platform

l Distribution or Replicationl Sync or Asyncl Transactionall Persists contents using a CacheLoader

− Write-through or write-behind− Shared or per cluster node

l Hibernate second-level cachel State of the art eviction strategies

Distributed Data Structure

Storage Engine (RAM

+Overflow)

Transaction

Query

Map/Reduce

Monitoring

Transport (JGroups)

Memcached Hotrod Rest

Node (JVM process)

Storage Engine (RAM

+Overflow)

Transaction

Query

Map/Reduce

Monitoring

Transport (JGroups)

Memcached Hotrod Rest

Node (JVM process)

TCP/UDP

RemoteClient

TCPTCP

High Availability•Memory is volatile•Make redundant copies

•Total replication (Replication Mode)•Partial replication (Distribution Mode)

• numOwners•Topology changes

•Nodes will crash!•Re-arrange state

Elasticity•Expect

•Node additions•Node removals

•Topology changes •are totally consistent•do not "stop the world"

API•Key-value store

• java.util.concurrent.ConcurrentHashMap•JSR-107 compliant•CDI support•Spring support

Integration•Hibernate integration

•2nd level cache•Lucene integration

•Query• index storage

•ModeShape•CapeDwarf•CacheStores

•Cassandra, MongoDB, ...

Key features•Transactions•Performance•Persistence•Map/Reduce•Querying

Big Data / Fast Data ?

Transactions•JTA transactions support

•XA or Synchronization based enlistment• recovery!

•Local transactions (batching)•Transactional modes

•optimistic •pessimistic• coming: ergonomics!

Persistence/Cache Store•Used for

•durability• increased storage•warm caches

•Various implementations•HBase, Cassandra, BDBJE, JDBC, MongoDB, File, Cloud,

Remote / Infinispan, ...•Extensible

•easy!

Network Performance & Reliability

Map/Reduce example

Map/Reduce

Map/Reduce

Infinispan on cloudsl Cloud-friendly autodiscovery strategies

l S3_PING, JDBC_PING, TCP_PINGl Fully dynamic clustering

l Add or kill nodes on the flyl Using a CacheLoader, scale back to zero nodes without data loss

Querying•Based on Hibernate Search/Lucene•Objects in the grid are indexed

• index management

Annotate your model@Indexedpublic class Book implements Serializable {

@Field String title; @Field String author; @Field String editor;

public Book(String title, String author, String editor) { this.title = title; this.author = author; this.editor = editor; }

}

Run a QuerySearchManager qf = Search.getSearchManager(cache); Query query = qf.buildQueryBuilderForClass(Book.class) .get() .phrase() .onField("title") .sentence("in action") .createQuery(); List<Object> list = qf.getQuery(query).list();

Use cases•Local cache•Cluster of caches•Autonomous data store

•access protocols

Local cache

Features of a local cache•Eviction•Expiry•Write through, write behind•Preloading•Notifications•Statistics

Local cache not good enough...

Cluster of caches

Limitations of embedded mode

•Client is affected by cache topology changes•Shared resources•Tier management

• incompatible JVM tuning•security•garbage collection

•Non-JVM clients

Cache Servers

Client/Server Protocols•REST•Memcached•Hotrod

• “proprietary”• java, python, ruby

Lucene Index Stored in Infinispan

Single Node Performance Idea

Multi-Node Setup

Performance Warnings• Set Lucene's maximum segment size to fit in LuceneDirectory chunk_size to avoid readlocks• Verify blob sizes fit in JGroups network packets• Check for CacheStores “sweet spot” size

The Path Ahead• 5.1.x.Final• 5.2 - February 2013

• non-blocking state transfer• cross-site replication• rolling upgrades for hotrod-clients• map/reduce marked stable

• 5.3 - Spring 2013• state transfer between sites

• 6.0• Beyond the current amazing performance• Scalable index writing• Advanced CacheStores

JBoss Data Grid (JDG)•JBoss Data Grid•Build on top of Infinispan•http://www.jboss.com/edg

Thank you!•infinispan.blogspot.com•@Infinispan•@SanneGrinovero