memory management for real-time java wes beebee and martin rinard laboratory for computer science...
TRANSCRIPT
Memory Management for Real-Time Java
Wes Beebee and Martin RinardLaboratory for Computer Science
Massachusetts Institute of Technology
Supported by: DARPA Program Composition for Embedded Systems
(PCES) Program
Goal: Enable Use of Java for Real-Time and Embedded
Systems
Vision
DownloadedJava Applets
Standard Java Applications
Real-TimeComputation
In Java
Unified Language/Environment Facilitates Interaction
Why Java?
• Type safe language, no memory corruption
• Reasonably modern approach• Object oriented• Garbage collected memory
management• Popular and supported…
• Programmers available• Tools available• Libraries available
Implications and Issues
• Heterogeneous components with different needs and goals• Real-time computation• User interface• Data management
• Issues• Memory management• Scheduling
•Event management and delivery•Processor allocation
Why NOT Java• Unpredictable memory usage
• Dynamic memory allocation• Allocation hidden in extensive set of
libraries and native methods• Allocation hidden in exception model
• Unpredictable execution times• Garbage collection • No scheduling guarantees
•Thread scheduling•Event delivery
• Complex libraries and native methods
Why NOT Java
• Impoverished set of abstractions• Threads, mutex locks, signal and wait• No good way express relationship between
•Events in system•Corresponding pieces of computation
• No good way to express timing expectations
• Real-Time Java Approach• Extend library• Native methods for new mechanisms
Real-Time Java Standard
Goal: Augment Java to better support real-time systems
• Augment memory model to enable threads to avoid garbage collection pauses
• Augment thread scheduling model to add more control over task scheduling
• Augment synchronization model to include lightweight event delivery mechanism
Our View• Real-time Java is a work in progress• Many of extensions generate
• More complex programming model• More possibilities for errors
• Our goal• Isolate general principles/concepts we
think will last• Develop new program analyses and
implementation mechanisms • That help programmers use real-time
extensions safely and effectively
Java Memory Models
• Java: single garbage-collected heap• Real-time Java: multiple kinds of
memories• Garbage-collected heap memory• Immortal memory (live for full
computation)• Scoped memories
(live for specific subcomputations)•Linear-time allocation (LTMemory)•Variable-time allocation (VTMemory)
Problems/Issues with Memory Model
• Scoped memory issues• Scoped memory reference checks• Scoped memory sizes
• Avoiding garbage collection interaction issues• No-heap real-time thread access
checks• Priority inversions caused by indirect
interactions with garbage collector
Scoped Memory Overview
StandardJava
Computation
Scoped Memory Overview
StandardJava
Computation
Objects in GC Heap
Scoped Memory Overview
StandardJava
Computation
Objects in GC Heap
Objects in GC Heap
Scoped Memory Overview
New ComputationTypically new thread
Maybe even real-time thread StandardJava
Computation
Objects in GC Heap
Scoped Memory Overview
New ComputationTypically new thread
Maybe even real-time thread StandardJava
Computation
ScopedMemory
New Thread RunsIn Scoped Memory
Objects in GC Heap
Scoped Memory Overview
New ComputationTypically new thread
Maybe even real-time thread StandardJava
Computation
ScopedMemory
Thread’s New ObjectsAllocated in
Scoped Memory
Objects
Objects in GC Heap
Scoped Memory Overview
New ComputationTypically new thread
Maybe even real-time thread StandardJava
Computation
ScopedMemory
Objects
Thread’s New ObjectsAllocated in
Scoped Memory
Objects in GC Heap
Scoped Memory Overview
New ComputationTypically new thread
Maybe even real-time thread StandardJava
Computation
ScopedMemory
Objects
Thread’s New ObjectsAllocated in
Scoped Memory
Objects in GC Heap
Scoped Memory Overview
New ComputationTypically new thread
Maybe even real-time thread StandardJava
Computation
ScopedMemory
Objects
ComputationTerminates
Objects in GC Heap
Scoped Memory Overview
New ComputationTypically new thread
Maybe even real-time thread StandardJava
Computation
ScopedMemory
Objects
Objects in Scoped MemoryDeallocated as a Unit
without GC
Scoped Memory Motivation
• Dynamic memory allocation without GC• Tie object lifetimes to computation lifetimes• Eliminate need to dynamically trace out
reachable objects• Warning:
• Example illustrates primary intended use• Specification allows more behaviors
•Scoped memories shared by multiple threads
•Nested scoped memories•Scoped memories entered multiple times
Safety Issue for Scoped Memories:
Dangling References• Lifetimes of objects in scoped memory
determined by lifetime of computation• Must ensure that no reference goes
from long-lived object to short-lived object
Nested Scoped Memories
ScopedMemory
Object
Referencing Constraints
ScopedMemory
ObjectReferencingDown Scopes
Is NOT OK
ReferencingUp Scopes
Is OK
Preventing Downward References
• Dynamic Reference Checks• At every write of a reference into an
object field or array element• Check that written object is allocated in
a scope with a lifetime at least as long as that of referred object
• If not, throw an exception• Drawbacks
• Dynamic checking overhead• New class of dynamic errors
Static Analysis
• Goal• Eliminate need for dynamic checks by• Statically checking that program does
not violate referencing constraints• Basic approach: escape analysis
What Escape Analysis Provides
void compute(d,e) ———— ———— ————
void multiplyAdd(a,b,c) ————————— ————————— —————————
void multiply(m) ———— ———— ————
void add(u,v) —————— ——————
Control Flow Graph• Nodes = methods• Edges = invocation relationships
What Escape Analysis Provides
void compute(d,e) ———— ———— ————
void multiplyAdd(a,b,c) ————————— ————————— —————————
void multiply(m) ———— ———— ————
void add(u,v) —————— ——————
Allocation Site
Control Flow Graph• Nodes = methods• Edges = invocation relationships
What Escape Analysis Provides
void compute(d,e) ———— ———— ————
void multiplyAdd(a,b,c) ————————— ————————— —————————
void multiply(m) ———— ———— ————
void add(u,v) —————— ——————
Allocation Site
Object Allocated HereDoes Not EscapeComputation of
multiplyAdd method
Control Flow Graph• Nodes = methods• Edges = invocation relationships
Our Escape Analysis• Interprocedural
• Analyzes interactions between methods• Recaptures objects in callers of allocating
methods• Compositional
• Analyzes each method once• Single analysis result that can be specialized
for use in different calling contexts• Suitable for multithreaded programs
• Analyzes interactions between threads• Recaptures objects that do not escape a
given multithreaded computation
Using Escape Analysis to Verify Correct Use of Scoped Memories
For each computation that runs in scoped memory
Check that allocated objects do not escape
Implementation
• FLEX compiler infrastructure (www.flexc.lcs.mit.edu)• Full Java compiler• Lots of utilities and packages• Support for deep program analyses
and transformations• Implemented scoped memories and
checks• Implemented escape analysis
• Used results to eliminate checks• In applications, eliminated all checks
Experimental Results
0
20
40
60
80
100
120
Array(Heap)
Array(Scope)
Tree(Heap)
Tree(Scope)
Water(Heap)
Water(Scope)
Barnes(Heap)
Barnes(Scope)
Benchmarks
Tim
e (
se
c)
Scope Checks
Application
Scoped Memory Sizes
• Scoped memory creation and sizeMemoryArea ma = new LTMemory(10000);“create a new scoped memory with 10,000 bytes”
• If try to allocate more than 10,000 bytes, implementation throws an exception
• Problems• Java does not specify object sizes• Size of given object may change during
its lifetime in computation• So how big to make scoped memory?
Objects in GC Heap
Modularity Problems
Scoped Memory SizeDetermined Here
StandardJava
Computation
ScopedMemory
Objects
Required Size Determined by
Behavior of Code in this
Computation
Objects in GC Heap
Modularity Problems
Scoped Memory SizeDetermined Here
StandardJava
Computation
ScopedMemory
Objects
• If change program, size may need to change!
•Amount of allocated memory becomes part of interface!
More Issues
• Different executions may allocate different amounts of data
• Lots of hidden allocation in libraries• Difficult to find out how much
memory is really allocated• If change implementation, may need
to change scoped memory size in clients
Analysis Solution
• Analyze program to symbolically compute allocated memory sizes• Input variables• Object sizes
• Compiler knows object sizes, can conservatively generate scoped memory sizes
Interaction with Garbage Collector
• Standard Collector Assumptions• Can interrupt computation at any point• Can suspend for unbounded time
• Real-Time Java extension• No-Heap Real-Time Threads• Can Access
•Immortal memory•Scoped memory
• Do not interact with GC heap AT ALL• Can run asynchronously with GC
Immortal
ScopedGC Heap
No-Heap Real-Time Thread Checks
• Dynamically check that no-heap real-time threads never access a location containing a reference into garbage-collected heap• At every read, check to make sure result
does not point into garbage-collected heap• At every write, check to make sure not
overwriting reference into GC heap• If check fails, throw exception
• Drawbacks• Dynamic checking overhead• New class of dynamic errors
Implementation
• FLEX compiler infrastructure (www.flexc.lcs.mit.edu)
• Implemented no-heap real-time threads• Implemented access checks• Measured performance with and without
checks
Experimental Results
0
50
100
150
200
250
300
Array Tree Water
Benchmark
Tim
e (s
ec)
Scope Checks
Heap ChecksApplication
Program Analysis for Eliminating Checks
• Control-flow analysis to identify code that may execute in no-heap real-time thread
• Global value-flow analysis • Tags each value that points to GC heap• Identifies all locations into which these
values may flow• Combine results
• Look at all no-heap real-time thread code• Check statically for access violations
Indirect Priority Inversions
Standard JavaThread
No-heap Thread
LockAcquire
Garbage Collector
LockAcquire
Interaction Between Resource Sharing and Garbage Collection
Indirect Priority Inversions
Standard JavaThread
No-heap Thread
LockAcquire
Garbage Collector
LockAcquire
Interaction Between Resource Sharing and Garbage Collection
Blocks
Indirect Priority Inversions
Standard JavaThread
No-heap Thread
LockAcquire
Garbage Collector
LockAcquire
No-heap thread must wait for standard Java thread to release lockStandard thread must wait for GC to finish (heap inconsistent until it finishes)No-heap thread must wait for GC!
Using Non-Blocking Synchronization to Eliminate Indirect Priority Inversions
Standard Java
Thread
No-heap Thread
StartAtomicRegion
Garbage Collector
StartAtomicRegion
EndAtomicRegion
DoesNot
Block! Abort,Retry
Implementation Status
• Non-blocking synchronization implemented for memory management primitives• Useful when threads share scoped
memory• Uses non-blocking synchronization
instructions from processor• Software implementation underway for
general atomic regions
Goal
•Dangling references for scoped memories•Resource needs of computations•Isolating computations from garbage collector
• Ensure threads with real-time constraints don’t access garbage collected data
• Eliminate indirect interactions•Our view
• Dynamic checks inadequate• Statically verify correct use, eliminate
checks
Enable safe real-time code to interact successfully with code that accesses GC
dataIssues and Complications
Broader View of Real-Time Java
• Java is best suited to express “batch” computations on objects• Not so good for control in
asynchronous, parallel, distributed, time-aware systems
• Inadequate for design/requirements• Can be part of solution, but only a part
Multiple Perspectives
• Any system has many desired properties• Data structure invariants• Flow of data between different
components• Timing requirements for computations
• Each property is inherently partial• Many properties are best expressed as
• Declarative constraints• NOT translatable into implementation
Design Conformance
Object ReferencingRelationships
Timing Constraints
Dataflow Interactions
Implementation
Check that implementation conforms
to design properties
Future
• Scheduling and event delivery• More precise referencing relationship analysis
• Completely characterize aliasing behavior• More flexible memory management
algorithms that preserve predictability• More flexible regions• Immediate deallocation
• Less restricted memory access constraints for real-time threads
• Design conformance for more control-oriented properties
Summary
• Real-time Java code coexists and interacts with standard Java code
• New complications (overhead + failure modes)• Scoped memory checks• Scoped memory sizes• No-heap real-time threads• Indirect priority inversions
• Attacked with program analysis• Future: scheduling and timing