memory management for real-time java wes beebee and martin rinard laboratory for computer science...

Memory Management for Real-Time Java

Wes Beebee and Martin RinardLaboratory for Computer Science

Massachusetts Institute of Technology

Supported by: DARPA Program Composition for Embedded Systems

(PCES) Program

Goal: Enable Use of Java for Real-Time and Embedded

Systems

Vision

DownloadedJava Applets

Standard Java Applications

Real-TimeComputation

In Java

Unified Language/Environment Facilitates Interaction

Why Java?

• Type safe language, no memory corruption

• Reasonably modern approach• Object oriented• Garbage collected memory

management• Popular and supported…

• Programmers available• Tools available• Libraries available

Implications and Issues

• Heterogeneous components with different needs and goals• Real-time computation• User interface• Data management

• Issues• Memory management• Scheduling

•Event management and delivery•Processor allocation

Why NOT Java• Unpredictable memory usage

• Dynamic memory allocation• Allocation hidden in extensive set of

libraries and native methods• Allocation hidden in exception model

• Unpredictable execution times• Garbage collection • No scheduling guarantees

•Thread scheduling•Event delivery

• Complex libraries and native methods

Why NOT Java

• Impoverished set of abstractions• Threads, mutex locks, signal and wait• No good way express relationship between

•Events in system•Corresponding pieces of computation

• No good way to express timing expectations

• Real-Time Java Approach• Extend library• Native methods for new mechanisms

Real-Time Java Standard

Goal: Augment Java to better support real-time systems

• Augment memory model to enable threads to avoid garbage collection pauses

• Augment thread scheduling model to add more control over task scheduling

• Augment synchronization model to include lightweight event delivery mechanism

Our View• Real-time Java is a work in progress• Many of extensions generate

• More complex programming model• More possibilities for errors

• Our goal• Isolate general principles/concepts we

think will last• Develop new program analyses and

implementation mechanisms • That help programmers use real-time

extensions safely and effectively

Java Memory Models

• Java: single garbage-collected heap• Real-time Java: multiple kinds of

memories• Garbage-collected heap memory• Immortal memory (live for full

computation)• Scoped memories

(live for specific subcomputations)•Linear-time allocation (LTMemory)•Variable-time allocation (VTMemory)

Problems/Issues with Memory Model

• Scoped memory issues• Scoped memory reference checks• Scoped memory sizes

• Avoiding garbage collection interaction issues• No-heap real-time thread access

checks• Priority inversions caused by indirect

interactions with garbage collector

Scoped Memory Overview

StandardJava

Computation

Scoped Memory Overview

StandardJava

Computation

Objects in GC Heap

Scoped Memory Overview

StandardJava

Computation

Objects in GC Heap

Objects in GC Heap

Scoped Memory Overview

New ComputationTypically new thread

Maybe even real-time thread StandardJava

Computation

Objects in GC Heap

Scoped Memory Overview

New ComputationTypically new thread

Maybe even real-time thread StandardJava

Computation

ScopedMemory

New Thread RunsIn Scoped Memory

Objects in GC Heap

Scoped Memory Overview

New ComputationTypically new thread

Maybe even real-time thread StandardJava

Computation

ScopedMemory

Thread’s New ObjectsAllocated in

Scoped Memory

Objects

Objects in GC Heap

Scoped Memory Overview

New ComputationTypically new thread

Maybe even real-time thread StandardJava

Computation

ScopedMemory

Objects

Thread’s New ObjectsAllocated in

Scoped Memory

Objects in GC Heap

Scoped Memory Overview

New ComputationTypically new thread

Maybe even real-time thread StandardJava

Computation

ScopedMemory

Objects

Thread’s New ObjectsAllocated in

Scoped Memory

Objects in GC Heap

Scoped Memory Overview

New ComputationTypically new thread

Maybe even real-time thread StandardJava

Computation

ScopedMemory

Objects

ComputationTerminates

Objects in GC Heap

Scoped Memory Overview

New ComputationTypically new thread

Maybe even real-time thread StandardJava

Computation

ScopedMemory

Objects

Objects in Scoped MemoryDeallocated as a Unit

without GC

Scoped Memory Motivation

• Dynamic memory allocation without GC• Tie object lifetimes to computation lifetimes• Eliminate need to dynamically trace out

reachable objects• Warning:

• Example illustrates primary intended use• Specification allows more behaviors

•Scoped memories shared by multiple threads

•Nested scoped memories•Scoped memories entered multiple times

Safety Issue for Scoped Memories:

Dangling References• Lifetimes of objects in scoped memory

determined by lifetime of computation• Must ensure that no reference goes

from long-lived object to short-lived object

Nested Scoped Memories

ScopedMemory

Object

Referencing Constraints

ScopedMemory

ObjectReferencingDown Scopes

Is NOT OK

ReferencingUp Scopes

Is OK

Preventing Downward References

• Dynamic Reference Checks• At every write of a reference into an

object field or array element• Check that written object is allocated in

a scope with a lifetime at least as long as that of referred object

• If not, throw an exception• Drawbacks

• Dynamic checking overhead• New class of dynamic errors

Static Analysis

• Goal• Eliminate need for dynamic checks by• Statically checking that program does

not violate referencing constraints• Basic approach: escape analysis

What Escape Analysis Provides

void compute(d,e) ———— ———— ————

void multiplyAdd(a,b,c) ————————— ————————— —————————

void multiply(m) ———— ———— ————

void add(u,v) —————— ——————

Control Flow Graph• Nodes = methods• Edges = invocation relationships

What Escape Analysis Provides

void compute(d,e) ———— ———— ————

void multiplyAdd(a,b,c) ————————— ————————— —————————

void multiply(m) ———— ———— ————

void add(u,v) —————— ——————

Allocation Site

Control Flow Graph• Nodes = methods• Edges = invocation relationships

What Escape Analysis Provides

void compute(d,e) ———— ———— ————

void multiplyAdd(a,b,c) ————————— ————————— —————————

void multiply(m) ———— ———— ————

void add(u,v) —————— ——————

Allocation Site

Object Allocated HereDoes Not EscapeComputation of

multiplyAdd method

Control Flow Graph• Nodes = methods• Edges = invocation relationships

Our Escape Analysis• Interprocedural

• Analyzes interactions between methods• Recaptures objects in callers of allocating

methods• Compositional

• Analyzes each method once• Single analysis result that can be specialized

for use in different calling contexts• Suitable for multithreaded programs

• Analyzes interactions between threads• Recaptures objects that do not escape a

given multithreaded computation

Using Escape Analysis to Verify Correct Use of Scoped Memories

For each computation that runs in scoped memory

Check that allocated objects do not escape

Implementation

• FLEX compiler infrastructure (www.flexc.lcs.mit.edu)• Full Java compiler• Lots of utilities and packages• Support for deep program analyses

and transformations• Implemented scoped memories and

checks• Implemented escape analysis

• Used results to eliminate checks• In applications, eliminated all checks

Experimental Results

0

20

40

60

80

100

120

Array(Heap)

Array(Scope)

Tree(Heap)

Tree(Scope)

Water(Heap)

Water(Scope)

Barnes(Heap)

Barnes(Scope)

Benchmarks

Tim

e (

se

c)

Scope Checks

Application

Scoped Memory Sizes

• Scoped memory creation and sizeMemoryArea ma = new LTMemory(10000);“create a new scoped memory with 10,000 bytes”

• If try to allocate more than 10,000 bytes, implementation throws an exception

• Problems• Java does not specify object sizes• Size of given object may change during

its lifetime in computation• So how big to make scoped memory?

Objects in GC Heap

Modularity Problems

Scoped Memory SizeDetermined Here

StandardJava

Computation

ScopedMemory

Objects

Required Size Determined by

Behavior of Code in this

Computation

Objects in GC Heap

Modularity Problems

Scoped Memory SizeDetermined Here

StandardJava

Computation

ScopedMemory

Objects

• If change program, size may need to change!

•Amount of allocated memory becomes part of interface!

More Issues

• Different executions may allocate different amounts of data

• Lots of hidden allocation in libraries• Difficult to find out how much

memory is really allocated• If change implementation, may need

to change scoped memory size in clients

Analysis Solution

• Analyze program to symbolically compute allocated memory sizes• Input variables• Object sizes

• Compiler knows object sizes, can conservatively generate scoped memory sizes

Interaction with Garbage Collector

• Standard Collector Assumptions• Can interrupt computation at any point• Can suspend for unbounded time

• Real-Time Java extension• No-Heap Real-Time Threads• Can Access

•Immortal memory•Scoped memory

• Do not interact with GC heap AT ALL• Can run asynchronously with GC

Immortal

ScopedGC Heap

No-Heap Real-Time Thread Checks

• Dynamically check that no-heap real-time threads never access a location containing a reference into garbage-collected heap• At every read, check to make sure result

does not point into garbage-collected heap• At every write, check to make sure not

overwriting reference into GC heap• If check fails, throw exception

• Drawbacks• Dynamic checking overhead• New class of dynamic errors

Implementation

• FLEX compiler infrastructure (www.flexc.lcs.mit.edu)

• Implemented no-heap real-time threads• Implemented access checks• Measured performance with and without

checks

Experimental Results

0

50

100

150

200

250

300

Array Tree Water

Benchmark

Tim

e (s

ec)

Scope Checks

Heap ChecksApplication

Program Analysis for Eliminating Checks

• Control-flow analysis to identify code that may execute in no-heap real-time thread

• Global value-flow analysis • Tags each value that points to GC heap• Identifies all locations into which these

values may flow• Combine results

• Look at all no-heap real-time thread code• Check statically for access violations

Indirect Priority Inversions

Standard JavaThread

No-heap Thread

LockAcquire

Garbage Collector

LockAcquire

Interaction Between Resource Sharing and Garbage Collection

Indirect Priority Inversions

Standard JavaThread

No-heap Thread

LockAcquire

Garbage Collector

LockAcquire

Interaction Between Resource Sharing and Garbage Collection

Blocks

Indirect Priority Inversions

Standard JavaThread

No-heap Thread

LockAcquire

Garbage Collector

LockAcquire

No-heap thread must wait for standard Java thread to release lockStandard thread must wait for GC to finish (heap inconsistent until it finishes)No-heap thread must wait for GC!

Using Non-Blocking Synchronization to Eliminate Indirect Priority Inversions

Standard Java

Thread

No-heap Thread

StartAtomicRegion

Garbage Collector

StartAtomicRegion

EndAtomicRegion

DoesNot

Block! Abort,Retry

Implementation Status

• Non-blocking synchronization implemented for memory management primitives• Useful when threads share scoped

memory• Uses non-blocking synchronization

instructions from processor• Software implementation underway for

general atomic regions

Goal

•Dangling references for scoped memories•Resource needs of computations•Isolating computations from garbage collector

• Ensure threads with real-time constraints don’t access garbage collected data

• Eliminate indirect interactions•Our view

• Dynamic checks inadequate• Statically verify correct use, eliminate

checks

Enable safe real-time code to interact successfully with code that accesses GC

dataIssues and Complications

Broader View of Real-Time Java

• Java is best suited to express “batch” computations on objects• Not so good for control in

asynchronous, parallel, distributed, time-aware systems

• Inadequate for design/requirements• Can be part of solution, but only a part

Multiple Perspectives

• Any system has many desired properties• Data structure invariants• Flow of data between different

components• Timing requirements for computations

• Each property is inherently partial• Many properties are best expressed as

• Declarative constraints• NOT translatable into implementation

Design Conformance

Object ReferencingRelationships

Timing Constraints

Dataflow Interactions

Implementation

Check that implementation conforms

to design properties

Future

• Scheduling and event delivery• More precise referencing relationship analysis

• Completely characterize aliasing behavior• More flexible memory management

algorithms that preserve predictability• More flexible regions• Immediate deallocation

• Less restricted memory access constraints for real-time threads

• Design conformance for more control-oriented properties

Summary

• Real-time Java code coexists and interacts with standard Java code

• New complications (overhead + failure modes)• Scoped memory checks• Scoped memory sizes• No-heap real-time threads• Indirect priority inversions

• Attacked with program analysis• Future: scheduling and timing