an engineer’s introduction to oracle coherence brian oliver senior principal solutions architect |...

<Insert Picture Here>

An Engineer’s Introduction to Oracle Coherence

Brian OliverSenior Principal Solutions Architect | Oracle

Agenda

• What is Coherence?• Demonstration• Consensus• How it works• Code Examples• Architectural Patterns

(c) Copyright 2007. Oracle Corporation

What is Coherence?

(c) Copyright 2007. Oracle Corporation

Oracle Coherence

• Development Toolkit• Pure Java 1.4.2+ Libraries• Pure .Net 1.1 and 2.0 (Client Libraries)• No Third-Party Dependencies• No Open Source Dependencies

• Other Libraries for…• Database and File System Integration• Top Link and Hibernate• Http Session Management, Spring, …

Oracle Coherence

• Provides…• Container-less Clustering of Java Processes• Data Structures to manage Data across a Cluster / Grid• Real-Time Event Observation – Listener Pattern• Materialized Views of Data • Parallel Queries and Aggregation – Object-based Queries• Parallel Data Processing• Parallel Grid Processing• RemoteException Free Distributed Computing• Clustered JMX• MAN + WAN Connectivity• Client + Data Grid Deployment Models

Oracle Coherence

• Some uses…• Application Clustering and Reliable Data Sharing• Caching state in the Application-tier• Relieve load on lower-tier systems

• Databases, Mainframes, Web Servers, Web Services

• Scaling out application state (in the application-tier)• In-Memory Http Session Management

• Resilient Processing Engine• Temporary System of Record for Extreme Transaction

Processing

Coherence Demonstration

(c) Copyright 2007. Oracle Corporation

How Coherence Works

(c) Copyright 2007. Oracle Corporation

Distributed Data Management (access)

(c) Copyright 2007. Oracle Corporation

The Partitioned Topology

(one of many)

In-Process DataManagement

Distributed Data Management (update)

(c) Copyright 2007. Oracle Corporation

Distributed Data Management (failover)

(c) Copyright 2007. Oracle Corporation

Distributed Data Management

• Members have logical access to all Entries• At most 2 network operations for Access• At most 4 network operations for Update• Regardless of Cluster Size• Deterministic access and update behaviour

(performance can be improved with local caching)

• Predictable Scalability• Cache Capacity Increases with Cluster Size• Coherence Load-Balances Partitions across Cluster• Point-to-Point Communication (peer to peer)• No multicast required (sometimes not allowed)

(c) Copyright 2007. Oracle Corporation

Data Distribution: Clients and Servers

(c) Copyright 2007. Oracle Corporation

“Clients” with storage disabled

“Servers” with storage enabled

Near Caching (L1 + L2) Topology

(c) Copyright 2007. Oracle Corporation

Observing Data Changes

(c) Copyright 2007. Oracle Corporation

Parallel Queries

(c) Copyright 2007. Oracle Corporation

Parallel Processing and Aggregation

(c) Copyright 2007. Oracle Corporation

Data Source Integration (read-through)

(c) Copyright 2007. Oracle Corporation

Data Source Integration (write-through)

(c) Copyright 2007. Oracle Corporation

Data Source Integration (write-behind)

(c) Copyright 2007. Oracle Corporation

Coherence Code Examples

(c) Copyright 2007. Oracle Corporation

Cluster cluster = CacheFactory.ensureCluster();Cluster cluster = CacheFactory.ensureCluster();

Clustering Java Processes

• Joins an existing cluster or forms a new cluster

• Time “to join” configurable

• cluster contains information about the Cluster

• Cluster Name

• Members

• Locations

• Processes

• No “master” servers

• No “server registries”

(c) Copyright 2007. Oracle Corporation

Leaving a Cluster

• Leaves the current cluster

• shutdown blocks until “data” is safe

• Failing to call shutdown results in Coherence having to detect process death/exit and recover information from another process.

• Death detection and recovery is automatic

(c) Copyright 2007. Oracle Corporation

CacheFactory.shutdown();CacheFactory.shutdown();

Using a Cacheget, put, size & remove

• CacheFactory resolves cache names (ie: “mine”) to configured NamedCaches

• NamedCache provides data topology agnostic access to information

• NamedCache interfaces implement several interfaces;

• java.util.Map, Jcache,ObservableMap*, ConcurrentMap*, QueryMap*, InvocableMap*

(c) Copyright 2007. Oracle Corporation

NamedCache nc = CacheFactory.getCache(“mine”);

Object previous = nc.put(“key”, “hello world”);

Object current = nc.get(“key”);

int size = nc.size();

Object value = nc.remove(“key”);

NamedCache nc = CacheFactory.getCache(“mine”);

Object previous = nc.put(“key”, “hello world”);

Object current = nc.get(“key”);

int size = nc.size();

Object value = nc.remove(“key”);

Coherence* Extensions

Using a CachekeySet, entrySet, containsKey

• Using a NamedCache is like using a java.util.Map

• What is the difference between a Map and a Cache data-structure?

• Both use (key,value) pairs for entries

• Map entries don’t expire

• Cache entries may expire

• Maps are typically limited by heap space

• Caches are typically size limited (by number of entries or memory)

• Map content is typically in-process (on heap)

(c) Copyright 2007. Oracle Corporation

NamedCache nc = CacheFactory.getCache(“mine”);

Set keys = nc.keySet();

Set entries = nc.entrySet();

boolean exists = nc.containsKey(“key”);

NamedCache nc = CacheFactory.getCache(“mine”);

Set keys = nc.keySet();

Set entries = nc.entrySet();

boolean exists = nc.containsKey(“key”);

Observing Cache ChangesObservableMap

• Observe changes in real-time as they occur in a NamedCache

• Options exist to optimize events by using Filters, (including pre and post condition checking) and reducing on-the-wire payload (Lite Events)

• Several MapListeners are provided out-of-the-box.

• Abstract, Multiplexing...

(c) Copyright 2007. Oracle Corporation

NamedCache nc = CacheFactory.getCache(“stocks”);

nc.addMapListener(new MapListener() {

public void onInsert(MapEvent mapEvent) {

}

public void onUpdate(MapEvent mapEvent) {

}

public void onDelete(MapEvent mapEvent) {

}

});

NamedCache nc = CacheFactory.getCache(“stocks”);

nc.addMapListener(new MapListener() {

public void onInsert(MapEvent mapEvent) {

}

public void onUpdate(MapEvent mapEvent) {

}

public void onDelete(MapEvent mapEvent) {

}

});

Querying CachesQueryMap

• Query NamedCache keys and entries across a cluster (Data Grid) in parallel* using Filters

• Results may be ordered using natural ordering or custom comparators

• Filters provide support almost all SQL constructs

• Query using non-relational data representations and models

• Create your own Filters

* Requires Enterprise Edition or above

(c) Copyright 2007. Oracle Corporation

NamedCache nc = CacheFactory.getCache(“people”);

Set keys = nc.keySet( new LikeFilter(“getLastName”, “%Stone%”));

Set entries = nc.entrySet( new EqualsFilter(“getAge”,

35));

NamedCache nc = CacheFactory.getCache(“people”);

Set keys = nc.keySet( new LikeFilter(“getLastName”, “%Stone%”));

Set entries = nc.entrySet( new EqualsFilter(“getAge”,

35));

Continuous ObservationContinuous Query Caches

• ContinuousQueryCache provides real-time and in-process copy of filtered cached data

• Use standard or your own custom Filters to limit view

• Access to “view”of cached information is instant

• May use with MapListeners to support rendering real-time local views (aka: Think Client) of Data Grid information.

(c) Copyright 2007. Oracle Corporation

NamedCache nc = CacheFactory.getCache(“stocks”);

NamedCache expensiveItems = new ContinuousQueryCache(nc, new GreaterThan(“getPrice”, 1000));

NamedCache nc = CacheFactory.getCache(“stocks”);

NamedCache expensiveItems = new ContinuousQueryCache(nc, new GreaterThan(“getPrice”, 1000));

Aggregating InformationInvocableMap

• Aggregate values in a NamedCache across a cluster (Data Grid) in parallel* using Filters

• Aggregation constructs include; Distinct, Sum, Min, Max, Average, Having, Group By

• Aggregate using non-relational data models

• Create your own aggregators

* Requires Enterprise Edition or above

(c) Copyright 2007. Oracle Corporation

NamedCache nc = CacheFactory.getCache(“stocks”);

Double total = (Double)nc.aggregate( AlwaysFilter.INSTANCE, new DoubleSum(“getQuantity”));

Set symbols = (Set)nc.aggregate( new EqualsFilter(“getOwner”, “Larry”), new DistinctValue(“getSymbol”));

NamedCache nc = CacheFactory.getCache(“stocks”);

Double total = (Double)nc.aggregate( AlwaysFilter.INSTANCE, new DoubleSum(“getQuantity”));

Set symbols = (Set)nc.aggregate( new EqualsFilter(“getOwner”, “Larry”), new DistinctValue(“getSymbol”));

Mutating InformationInvocableMap

• Invoke EntryProcessors on zero or more entries in a NamedCache across a cluster (Data Grid) in parallel* (using Filters) to perform operations

• Execution occurs where the entries are managed in the cluster, not in the thread calling invoke

• This permits Data + Processing Affinity

* Requires Enterprise Edition or above

(c) Copyright 2007. Oracle Corporation

NamedCache nc = CacheFactory.getCache(“stocks”);

nc.invokeAll( new EqualsFilter(“getSymbol”, “ORCL”), new StockSplitProcessor());

...

class StockSplitProcessor extends AbstractProcessor {

Object process(Entry entry) { Stock stock = (Stock)entry.getValue(); stock.quantity *= 2; entry.setValue(stock); return null; }

}

NamedCache nc = CacheFactory.getCache(“stocks”);

nc.invokeAll( new EqualsFilter(“getSymbol”, “ORCL”), new StockSplitProcessor());

...

class StockSplitProcessor extends AbstractProcessor {

Object process(Entry entry) { Stock stock = (Stock)entry.getValue(); stock.quantity *= 2; entry.setValue(stock); return null; }

}

Oracle Coherence Architectural Patterns

(c) Copyright 2007. Oracle Corporation

Single Application Process

(c) Copyright 2007. Oracle Corporation

Clustered Processes

(c) Copyright 2007. Oracle Corporation

Multi Platform Cluster

(c) Copyright 2007. Oracle Corporation

Clustered Application Servers

(c) Copyright 2007. Oracle Corporation

With Data Source Integration(Cache Stores)

(c) Copyright 2007. Oracle Corporation

Clustered Second Level Cache(for Hibernate)

(c) Copyright 2007. Oracle Corporation

Remote Clients connected toCoherence Cluster

(c) Copyright 2007. Oracle Corporation

Interconnected WAN Clusters

(c) Copyright 2007. Oracle Corporation

GettingOracle Coherence

(c) Copyright 2007. Oracle Corporation

Oracle Coherence

• Search: • http://search.oracle.com

• Download• http://www.oracle.com/technology/products/coherence

• Support• http://forums.tangosol.com• http://wiki.tangosol.com

• Read More• http://www.tangosol.com/

CoherenceSearch For:

Questions

(c) Copyright 2007. Oracle Corporation

Appendix

The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions.The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.