Riding The Memory Bus

Ophir Radnitz

Agenda

• RAM vs. IO

• Tools of the Trade

• Introducing Terracotta

• RAM Patterns with Terracotta

Anatomy of an Application

Network

Access disk,Serialization

Start a request Get a DB connection, start transactioning

Get a bunch of data

Release resources

Render stuff

I/O Issues

• Scalability issues• Excessive network chatter• How do you scale the database?

• Only as fast as your slowest query• Implies serialization/deserialization

Bandwidths (1 of 2)

Device BandwidthRAMFSB 1,333MHz, DDR3 10 GB/s

PCI Express x16 8 GB/sDMI Interface 1 GB/sSATA 300 MB/sGigabit ethernet 100 MB/sSustained Disk IO(minimal seeks) ~60 MB/s

Bandwidths (1 of 2)

Device BandwidthFirewire 800 ~55 MB/sUSB 2.0 ~30 MB/s100 Mbit ethernet 10 MB/s10 Mbit ethernet 1 MB/sInternet(8Mbit down/1Mbit up)

1.1 MB/s down0.17 MB/s up

RAM vs. Disk

RAM DiskCo-located with CPU Shared storage

Mapped (Indexed) Sequential storage (stream of bytes)

Average access: ~83 ns

Read Seek ~13.7msWrite seek ~15ms

Internet: ~80ms

RAM Goodness

• RAM = High Bandwidth & Low

Latency

• No serialization costs

• Less network

Use Cases

• Low level cache• SQL, HTTP

• Application cache• Hibernate 2nd level cache

• Short lived entities• Queues

Generic Tools

• Memcached• A high-performance, distributed memory

caching system• Used EVRYWHERE: Facebook, Twitter,

Digg, Wikipedia, Slashdot, LiveJournal, Sourceforge etc.

• Squid• Routing and load balancing • HTTP cache

Memcached Limitations for Java

• Remote cache

• Serialization

• 2Gb limit

Java In-Memory Tools

• Prevayler, Space4J• Persist objects in memory, supports

ACID and file system journaling• Db4O in memory, Perst

• Object oriented database• Jofti

• A high-performance object indexing and searching solution. Supports Map.

In-Memory Considerations

• Reliability requires redundancy• And/or durability

• In a cluster state must be synced

• Requires state distribution

Java Distribution Tools

• Coherence• GigaSpace• Ehcache• Infinispan• Terracotta• Hazelcast

Introducing Terracotta

• Network-Attached Memory infrastructure for the JVM

• Open source• (TPL, based on Mozilla License)

• Current version: 3.0.1

Terracotta Customers

Introducing Terracotta

• Highly Available• Highly Scalable• Avoids excessive state

replication• Hub and spoke architecture

A Nice Diagram

Application

ApplicationApplication

Application

Application

TerracottaServer

TerracottaServer

TerracottaServer

How Terracotta Works

Application

TerracottaServer

JVM

Terracotta Libraries

Bootjar instrumenting your application

TC handles JVM locks and

data access

TerracottaClient

How Terracotta Works, take 2

Terracotta Server

Application Application Application

Actual instance

Reference

Terracotta Features

• Cross-JVM object identity • Automatic Persistence to disk• Virtual Memory

• spill heap to TC and/or to disk, transparently• Cluster profiling & visualization• JMX-based monitoring and

management

Terracotta Advantages

• Easy to integrate, Maven support• Great scalability• No proprietary API (optional)• Ready made integration modules• Great diagnostics• In/out of process

Terracotta Development Console

Integrating Terracotta

1. Download & extract

2. Write a tc-config.xml

3. Create a bootjar

4. Run your app with the bootjar

5. Add elements & refactor

TIM - Terracotta Integration Modules

• Packaged functionality• Easy to integrate

• Handles locks details• Examples:

• Concurrent collections, Ehcache• Lucene/Compass, Spring Security• Pipes, Wicket, Tomcat, Jetty • Hibernate, JBoss, Spring etc.

Terracotta Elements

• HTTP Session Replication• Cache Evictor• Queues• Master-Worker• Asynchronous Processor• Cluster membership events

Terracotta Patterns

• Data Cache

• Intermediate persistence

• Queues / work distribution

• Write behind to database

Data Cache Considerations

• Figure out data lifecycle• Read-only, metadata, configuration, session

• Where cache is applicable• Single point of update

• Set eviction policies• LRU, FIFO

Data Cache

• Session cache is easier, entity cache needs real distribution

• Use a generic cache• ConcurrentMap + Map Evictor (+ Jofti)

• Or a Hibernate 2nd level cache• ehcache, TC-Cache

Where Should My Objects Live?Ap

prop

riat

enes

s

Data Lifetime

Memory Database

dies quickly stays forever

Intermediate Data Lifetime

• Some data doesn’t have to be stored in the database

• Session information• Messages / Tokens• Transaction• Unfinished stuff

Data Lifetime

TIM Pipes: Queues

• Execute time consuming tasks asynchronously• In a reliable way• Load balanced

• Examples:• Send emails, process images

• Supports pub/sub, routing• No need for JMS

CommonJ – Work Distribution

• A joint API spec by Oracle and IBM• Provides interfaces for:

• Worker / WorkItem / WorkManager• Scheduler• etc…

• TIM-Messaging provides an implementation• Pipes• Master / Worker

Write Behind to System Of Record

• Asynchronously persist to DB• Work queue• “1.5 phase commit”

• Flags objects as dirty• Avoids syncing dirty objects• Flushes objects• Flags objects as clean

Terracotta Drawbacks

• Concurrency awareness• Massive scalability is a commercial

feature• Requires another machine(s)• Sharing objects across applications

requires configuration• TIMs documentation

Terracotta Pros

• Concurrency awareness• Fits into your application• Provides great visibility• Very versatile usage• Scales massively• Doesn’t use serialization• Great community

Summary

• Does it belong in the DB?

• Do I have to go to the DB for that?

• Do I have to do it now?

Ask yourself:

Terracotta Pros

Any Questions?