Enabling Java for Persistent Memory

October 3, 2017

Any mention of Intel compilers, libraries or Intel products containing Intel compilers or libraries. Note that libraries include, but are NOT limited to, the Intel® Threading Building Blocks (Intel® TBB), Intel® Math Kernel Library (Intel® MKL), Intel® Integrated Performance Primitives (Intel® IPP), Intel® Media SDK and Intel® MPI Library.

Performance of compiled code, including benchmarks, when Intel compilers or libraries were involved.

When in doubt, include the “optimization notice” slide.

This applies to any presentation in which Intel is involved. This means any presentation in which is completely or partially created by, or presented by, Intel.

Do not change the text in a disclaimer.

You may refer any questions regarding this to Intel Legal, or Intel SSG software development products marketing. As of March 2014, James Reinders (Intel SSG DPD) and Mike Sirtori (Intel Legal) are key contacts who will know how to help with an inquiry.

���2

Intel® Persistent Memory (3D XPoint)

New type of memory:

• Persistent

• 6 TB per two-socket system

• Cheaper than DRAM

Product available:

• SSDs in 2017

• Intel DIMMs for next-gen platforms in 2018

▪ Byte Addressable

▪ Load/store access

3

• Assume a heterogeneous memory architecture

• Take advantage of lower cost and higher capacity

• Two usage models

▪ Whole Java Heap on PM

▪ Heterogeneous Java Heap

Java enabling for Volatile Use of Persistent Memory

4

Whole Java Heap on PM

• Allocation of Java heap on new kinds of memories

• Use case:

▪ In multi-JVM deployments to prioritize Java VMs. Example: Oracle Fusion Apps

▪ Applications which can benefit from large memory. Example: big-data applications and in-memory databases

• Proposed JEP on OpenJDK is under review.

https://bugs.openjdk.java.net/browse/JDK-8171181

5

Whole Java Heap on PM

• Heap memory allocation on Persistent memory

• Allocating Java heap on non-DRAM memory

java -Xmx32g -Xms16g -XX:HeapDir=/XPointFS/heap ApplicationClass

Heap memory Non-heap memory

DRAMPersistent Memory

Ext4 DAX File System

mmap(…, file)

JVM

6

▪ Support within Java VM for a heterogeneous heap

▪ Large or infrequently accessed objects reside in slower memory

▪ Frequently accessed objects reside in DRAM

▪ User directed allocation, OR,

▪ Automatic object movement based on access profile.

Java Heterogeneous Heap

7

Prototype

▪ Two kinds of HeapAreas

• NORMAL or DRAM HeapArea

• COLD HeapArea

▪ User directed allocation

▪ Default is always Normal (DRAM) HeapArea

▪ Change the memory management module of JVM

▪ Garbage collection (G1 GC) collects both HeapAreas

NORMAL HeapAreaCOLD

HeapArea

Java Heap

MCDRAM PMDRAM

8

Prototype : interface

• COLD HeapArea size specified using flag –Xmp

Example : -Xmx60g -Xmp500g

• APIs for setting allocation context

▪ Deep cloning using Kryo.

MyObj_on_CR = Kryo.copy_to_PM(MyObj);

// Set the allocation target to PM

Heap.setAllocationTarget(AllocationTarget.PM);

HashMap user_records = new HashMap(1000000);

// reset allocation target to DRAM.

Heap.resetAllocationTarget();

DRAM

PM

Java Heap

9

Big Data Case Study - Apache Spark

▪Framework for cluster computing

▪ In memory computation

▪Benefits from larger memory capacity

▪Goal : avoid need to use Disk

10

New RDD Storage Level – MEMORY_AND_PM

▪Uses Java prototype with COLD HeapArea

▪DRAM acts as level 1 cache

▪Overflow partitions saved in PM

▪Application only needs to change storage level

11

Summary

▪Persistent memory – volatile usage

▪Whole heap : Coarse granularity

▪Heterogeneous heap : Finer granularity

▪Big data applications

12

Persistent Collections for Java Project

1. Library of persistent classes ▪ state stored on persistent heap ▪ instances behave like regular Java objects, just longer-lived ▪ reachability-based lifetime ▪ easy-to-understand data consistency model ▪ no serialization or deserialization

2. API for defining persistent classes ▪ expressiveness similar to that of regular classes ▪ no change to developer toolchain

3. Low-level accessor API (MemoryRegion) ▪ developer can roll their own abstractions

13

https://github.com/pmem/pcj

Persistent Collections -- Uses

1. In databases or data grids, replace or augment serialized storage with persistent collections ▪ e.g. customer wants to replace Cassandra storage layer

▪ Goal is to be faster and simpler than current solutions 2. Replace simple database use with persistent collections 3. Long-lived caches / memoization ▪ Lookups, initialization, function results, other computations

▪ Profitable where volatile cache or serialization is not 4. Innovation enabled by persistence capability

14

Library of Persistent Classes

▪ Primitive arrays (e.g. PersistentByteArray, mutable and immutable)

▪ PersistentArray<E extends PersistentObject> (mutable and immutable)

▪ PersistentTuple<T1 extends PersistentObject, …> (mutable and immutable)

▪ PersistentArrayList<E extends PersistentObject>

▪ PersistentHashMap<K extends PersistentObject, V extends PersistentObject>

▪ PersistentLinkedList<E extends PersistentObject>

▪ PersistentLinkedQueue<E extends PersistentObject>

▪ PersistentSkipListMap<K extends PersistentObject, V extends PersistentObject>

▪ PersistentFPTreeMap<K extends PersistentObject, V extends PersistentObject>

▪ PersistentSIHashMap<K extends PersistentObject, V extends PersistentObject>

▪ ObjectDirectory - indefinitely reachable root map of <String, T extends PersistentObject>

▪ ------------------------------------------------------------------------------------------------------------

▪ Primitive types (as field and array element values, no separate class)

▪ Boxed primitives (e.g. PersistentLong)

▪ PersistentString

▪ PersistentByteBuffer

▪ PersistentAtomicReference<T extends PersistentObject>

15

PersistentIntArray a = new PersistentIntArray(1024); // Ints are allocated on persistent heap a.set(0, 123); // 4-byte int value written to persistent heapa = null; // Array is unreachable. Object will be collected

PersistentIntArray data = new PersistentIntArray(1024);ObjectDirectory.put("Application_data", data); // no serialization, reference to array is writtendata.set(0, 123);

// restart

PersistentIntArray data1 = ObjectDirectory.get("Application_data", PersistentIntArray.class);assert(data.get(0) == 123);

public class Application { static PersistentIntArray data;

static { data = ObjectDirectory.get("Application_data", PersistentIntArray.class); if (data == null) ObjectDirectory.put("Application_data", data = new PersistentIntArray(1024)); }

// ...}

0102030405060708091011121314151617181920212223242526272829

Using persistent classes

ObjectDirectory idiom

16

public class Employee extends PersistentTuple2<PersistentLong, PersistentString> { public Employee(PersistentLong id, PersistentString name) { setId(id); setName(name); }

public PersistentLong getId() { return _1(); } public void setId(PersistentLong id) { _1(id); } public PersistentString getName() { return _2(); } public void setName(PersistentString name) { _2(name); }

public String toString() { return String.format("Employee(%s, %s)", getId(), getName()); }}

0102030405060708091011121314151617181920212223242526

Extend a built-in class with code

17

PersistentArrayList<PersistentString> movies = new PersistentArrayList<>();PersistentArrayList<PersistentString> movieIndex = new PersistentArrayList<>();

public void addMovie(PersistentString movie) { Transaction.run(() -> { movies.add(movie); movieIndex.add(movie); }); }

// transaction groups allocation and useTransaction.run(() -> { PersistentString m = new PersistentString("Jaws"); addMovie(m);}

// or garbage collection rolls back allocation -- no leakPersistentString m = new PersistentString("Jaws");// crash hereaddMovie(m);

void incRefCount() { MemoryRegion region = getPointer().region(); Transaction.run(() -> { long offset = Header.TYPE.getOffset(Header.REF_COUNT); region.putInt(offset, region.getInt(offset) + 1); }, this);}

0102030405060708091011121314151617181920212223242526272829303132333435

Transactions

Internal use of Transactions

Transaction idioms

18

public final class Employee { private final long id; private String name;

public Employee(long id, String name) { this.id = id; setName(name); }

public long getId() {return id;} public String getName() {return name;} public void setName(String name) {this.name = name;}

public int hashCode() {return Long.hashCode(getId());}

public boolean equals(Object obj) { if (!(obj instanceof Employee)) return false; Employee emp = (Employee)obj; return emp.getId() == getId() && emp.getName().equals(getName()); }

public String toString() { return String.format("Employee(%d, %s)", getId(), getName()); }}

01020304050607080910111213141516171819202122232425262728293031

Non-persistent

19

public final class Employee extends PersistentObject { private static final LongField ID = new LongField(); private static final StringField NAME = new StringField(); private static final ObjectType<Employee> TYPE = ObjectType.withFields(Employee.class, ID, NAME);

public Employee(long id, PersistentString name) { super(TYPE); setLongField(ID, id); setName(name); }

private Employee(ObjectPointer<Employee> p) {super(p);}

public long getId() {return getLongField(ID);} public PersistentString getName() {return getObjectField(NAME);} public void setName(PersistentString name) {setObjectField(NAME, name);} public int hashCode() {return Long.hashCode(getId());}

public boolean equals(Object obj) { if (!(obj instanceof Employee)) return false; Employee emp = (Employee)obj; return emp.getId() == getId() && emp.getName().equals(getName()); }

public String toString() { return String.format("Employee(%d, %s)", getId(), getName()); }}

01020304050607080910111213141516171819202122232425262728293031

Persistent

20

Interface-based Class Definition

▪ An alternate to the meta-field declaration scheme ▪ Post from John Rose (Oracle):

▪ http://cr.openjdk.java.net/~jrose/panama/using-interfaces.html

▪ Pass an interface to a "binder" (e.g. Persistent) ▪ Get back a factory-like object ▪ Instances implement the interface and their

behavior is consistent with the binder's name (e.g. are persistent)

▪ Excited by early proof-of-concept work

21

Low-level: MemoryRegions

▪ Interface from OpenJDK Panama project ▪ Get and set for byte, short, int, long (on persistent memory) ▪ We add a Heap API to allocate and free MemoryRegions ▪ Lets developers: ▪ retrofit existing code at low-level ▪ create their own abstractions

▪ Three versions are implemented: 1. RawMemoryRegion -- useful for volatile use or when caller provides

data consistency externally 2. FlushableMemoryRegion -- includes flush() method and fail-safe

isFlushed() state 3. TransactionalMemoryRegion -- writes are transactional

22

Backup

23

Built-in Persistent Classes

Memory Hardware

PersistentMemoryProvider

Transactions MemoryRegions

Developer code (incl. new Persistent Classes)

OS

Implementation Stack

Native code (via JNI)

Java VM

Java with native methods

pure Java

Non-Volatile Memory Library (NVML)

native code

24

Use of Non-Volatile Memory Library (NVML)

▪We use NVML for:

▪ Provisioning pools

▪Memory allocation

▪ Reading and writing bytes

▪ Core transaction support

▪We call C code via JNI get to the above functionality in NVML

▪ This project's code is mostly Java (>90%) but we depend on NVML for persistence

25

https://github.com/pmem/nvml

8 Things to Get Right

1. APIs 2. Data consistency 3. Object lifetime 4. Concurrency 5. Performance 6. Security 7. Portability 8. Object / class evolution

26