sovereign: migrating java threads to improve availability of web applications
DESCRIPTION
Transparent failover as a technique to improve reliability of mission critical systems is not usable with web applications that rely on additional threads in their HTTP sessions. RAP applications fall into this category of web applications. Sovereign is a research project by EclipseSource in conjunction with the Technical University Munich to develop a solution for the above problem. In this session we discuss two solutions under development in the Sovereign project. The first solution utilizes byte code weaving with AspectJ in order to connect the RAP application with Terracotta, a distributed heap for Java objects. The second solution being explored is to extend the Maxine VM by a feature to replicate/migrate objects and threads between VM instances. We describe their advantages and disadvantages for overcoming the problem of immobile Java threads that prevent transparent failover for RAP applications. We demonstrate how to transfer the state of a running RAP application from one server to another using both methods.TRANSCRIPT
eclipsesource-logo
Strategies Recording Modified VM Outlook
Sovereign:Migrating Java Threads
to Improve Availability of Web Applications
Johannes Eickhold, Technical University MunichMarkus Knauer, EclipseSource
Dr. Thomas Fuhrmann, Technical University Munich
March 22, 2011
[email protected] Sovereign March 22, 2011 1 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook
What is high availability for web applications?
Clu
ste
r
Servlet Container
Load Balancer
Clu
ste
r
Servlet Container
Load Balancer
Every node can take overOnly works if app is stateless
[email protected] Sovereign March 22, 2011 2 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook
What is RAP?
RAP supportsReusing RCP’s UI thread→ single sourcingJob framework
[email protected] Sovereign March 22, 2011 3 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook
Challenge: Transfer Server Side State
Servlet Container
Clu
ste
r
Load Balancer
UIT
hre
ad
UIT
hre
ad
SS S
UIT
hre
ad
sticky sessions!no-fail-over = true
GoalEnable failsave operation of all RAP features
[email protected] Sovereign March 22, 2011 4 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook
Overview
1 Strategies
2 RecordingStrategy ExplainedExampleDemo
3 Modified VMStrategy ExplainedDemo
4 Outlook
[email protected] Sovereign March 22, 2011 5 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook
Overview
1 Strategies
2 RecordingStrategy ExplainedExampleDemo
3 Modified VMStrategy ExplainedDemo
4 Outlook
[email protected] Sovereign March 22, 2011 6 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook
Two Strategies for Enabling Session Failover
1 Recording StrategyRecords the widget state to shared heapOperates on application level
2 Modified VM StrategyEnables thread migration on the JVM level
[email protected] Sovereign March 22, 2011 7 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook
Recording Strategy
[email protected] Sovereign March 22, 2011 8 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook
Modified VM Strategy
replicate threads
primary VM secondary VM
freeze thread + replicate state and data + continue thread =thread migration
[email protected] Sovereign March 22, 2011 9 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook
Recording Strategy vs. Modified VM Strategy
record replay
UIThread UIThread
byte code weaving unmodified applicationunmodified Hot Spot VM enhanced Maxine VMTerracotta shared heap object and thread replication
[email protected] Sovereign March 22, 2011 10 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook Strategy Explained Example Demo
Overview
1 Strategies
2 RecordingStrategy ExplainedExampleDemo
3 Modified VMStrategy ExplainedDemo
4 Outlook
[email protected] Sovereign March 22, 2011 11 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook Strategy Explained Example Demo
Recording Strategy
Modifications to existing RAP framework1 Record widget creation order on primary2 Replay recorded widgets during creation phase on
secondary3 Cluster the session including widget tree4 Remove non-serializable UI thread from session
Incrementally add failover support for additional RAP widgets
[email protected] Sovereign March 22, 2011 12 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook Strategy Explained Example Demo
Lifecycle of a RAP application
Shared Heap
Client
UI Thread
request
Initial request creates the UI thread
[email protected] Sovereign March 22, 2011 13 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook Strategy Explained Example Demo
Lifecycle of a RAP application
Shared Heap
Client
UI Thread
request
UI thread executes createUI() and creates widget graph
[email protected] Sovereign March 22, 2011 13 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook Strategy Explained Example Demo
Lifecycle of a RAP application
record
Shared Heap
Client
UI Thread
request
The widget graph is stored on the shared heap
[email protected] Sovereign March 22, 2011 13 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook Strategy Explained Example Demo
Lifecycle of a RAP application
response
Shared Heap
Client
UI Thread
Servlet responds with rendered UI to client.
[email protected] Sovereign March 22, 2011 13 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook Strategy Explained Example Demo
Hot Standby Takes Over
Shared Heap
Client
FAILURE
UI Thread
request
Primary servlet container fails and secondary jumps in
[email protected] Sovereign March 22, 2011 14 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook Strategy Explained Example Demo
Hot Standby Takes Over
UI Thread Shared Heap
Client
request
UI Thread
FAILURE
Primary servlet container fails and secondary jumps in
[email protected] Sovereign March 22, 2011 14 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook Strategy Explained Example Demo
Hot Standby Takes Over
UI Thread Shared Heap
Client
request
UI Thread
FAILURE
Skip widget creation in new UI thread!
[email protected] Sovereign March 22, 2011 14 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook Strategy Explained Example Demo
Failover Situation
replay
UI Thread Shared Heap
Client
request
UI Thread
FAILURE
Widgets are restored from the shared heap
[email protected] Sovereign March 22, 2011 15 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook Strategy Explained Example Demo
Failover Situation
response
UI Thread Shared Heap
Client
UI Thread
FAILURE
We achieved transparent failover!
[email protected] Sovereign March 22, 2011 15 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook Strategy Explained Example Demo
Implementation
You could implement this yourselfor
reuse existing technologies.
AspectJ is used to:Record order in which widgets are createdTrack method calls to enable method skipping
Terracotta provides a shared heap
[email protected] Sovereign March 22, 2011 16 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook Strategy Explained Example Demo
Recap of Recording Strategy
ShareTIMs: Equinox,JettyConfigure RAP(tc-config.xml)
RecordTracing widgetcreation withAspectJSkip methodson secondary
No silver bullet!RAP application has to be designed for failover
[email protected] Sovereign March 22, 2011 17 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook Strategy Explained Example Demo
Jump to the example
Example RAP application: "swt layout"Aspects implementing the recording/replay
[email protected] Sovereign March 22, 2011 18 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook Strategy Explained Example Demo
SWT Layout Example
1 Label l = new Label(shell, SWT.NONE);2 l.setText("relocate me!");3 ...4 Button b = new Button(shell, SWT.PUSH);5 b.setText("left");6 b.addSelectionListener(new SelectionAdapter() {7 public void widgetSelected(SelectionEvent e) {8 localCounter ++;9 globalCounter++;
10 l.setText("relocated "11 + localCounter + "/" + globalCounter12 + " times");13 }14 });
[email protected] Sovereign March 22, 2011 19 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook Strategy Explained Example Demo
Trace RAP Widget Creation and Method Calls
UIThread Button shared heap
new()
store
setText()
update
On primary
UIThread Button shared heap
new()
restore
setText()
On secondary
[email protected] Sovereign March 22, 2011 20 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook Strategy Explained Example Demo
Recording Demo
[email protected] Sovereign March 22, 2011 21 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook Strategy Explained Demo
Overview
1 Strategies
2 RecordingStrategy ExplainedExampleDemo
3 Modified VMStrategy ExplainedDemo
4 Outlook
[email protected] Sovereign March 22, 2011 22 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook Strategy Explained Demo
Modified VM Strategy
Extensions to the Maxine research VMObject replicationThread replication
All VMs in web cluster execute:
MigrationAgent + target application
[email protected] Sovereign March 22, 2011 23 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook Strategy Explained Demo
Maxine VM Internals
Maxine VM is written in Java (metacircular)
ThreadsUses pthreadsOne-on-one mapping: Java↔ pthreadSafepoint operations inserted at backward branchesFreezing of threads only via VmOperation
CompilersBaseline: template JITOptimizing: port of C1X from HotSpot or CPS
Details in wiki: http://wikis.sun.com/display/MaxineVM
[email protected] Sovereign March 22, 2011 24 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook Strategy Explained Demo
Migration via Agents
RAP clusteringAgents run next to Equinox in same VMReplication only used for UI threads of RAP sessions
migrationAgent.migrate(uiThread);
[email protected] Sovereign March 22, 2011 25 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook Strategy Explained Demo
Continuous Thread Replication
On primary:
Client RequestThread Replicator UIThread Secondary
HTTP request
handle(request)
notify(request)
handle(request)
finished
replicate(UIThread)
frames & objects
HTTP response
[email protected] Sovereign March 22, 2011 26 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook Strategy Explained Demo
Replication of Stack Frames
On secondary:
Thread
receivingmigration
agent
VmThread.run()
ThreadStub.run()
nativeThreadCreate()
receivesreplicated thread
preparewait
stack of ThreadStub
[email protected] Sovereign March 22, 2011 27 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook Strategy Explained Demo
Replication of Stack Frames
On secondary:
receivesdebug information
receivingmigration
agentcreate
& rewire
stack buffer
...
MigratedThread.run()
method1()
method2()
methodN()
frame n frame 2 frame 1...
[email protected] Sovereign March 22, 2011 27 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook Strategy Explained Demo
Replication of Stack Frames
On secondary:
receivingmigration
agent
stack of ThreadStub
resize
nativeThreadCreate()
VmThread.run()
ThreadStub.run()
frame n frame 2 frame 1...
receivesdebug information
[email protected] Sovereign March 22, 2011 27 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook Strategy Explained Demo
Replication of Stack Frames
On secondary:
receivingmigration
agent
stack of ThreadStub
resize
nativeThreadCreate()
VmThread.run()
ThreadStub.run()
frame n frame 2 frame 1...
receivesdebug information
[email protected] Sovereign March 22, 2011 27 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook Strategy Explained Demo
Replication of Stack Frames
On secondary:
...
method2()
method1()
MigratedThread.run()
methodN()
receivingmigration
agent bufferinsert
stack of ThreadStub
nativeThreadCreate()
VmThread.run()
ThreadStub.run()
frame n frame 2 frame 1...
receivesdebug information
[email protected] Sovereign March 22, 2011 27 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook Strategy Explained Demo
Replication of Stack Frames
On secondary:
ThreadStub.patch()
methodN()
method2()
method1()
MigratedThread.run()
...
receivingmigration
agent rewire
stack of ThreadStub
nativeThreadCreate()
VmThread.run()
ThreadStub.run()
frame n frame 2 frame 1...
receivesdebug information
[email protected] Sovereign March 22, 2011 27 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook Strategy Explained Demo
Inspecting the Migration Algorithm
RCP based tooling:Inspect migrated entities at VM level
[email protected] Sovereign March 22, 2011 28 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook Strategy Explained Demo
Migration Demo
[email protected] Sovereign March 22, 2011 29 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook
Overview
1 Strategies
2 RecordingStrategy ExplainedExampleDemo
3 Modified VMStrategy ExplainedDemo
4 Outlook
[email protected] Sovereign March 22, 2011 30 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook
Conclusion
Clustering RAP’s UIThreads is possible
Achieves transparent failover
[email protected] Sovereign March 22, 2011 31 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook
Future Work
1 Future directions for clustered RAP:Support more SWT widgetsJFace and WorkbenchSupport alternatives to Terracotta
2 Potential for Java VMs with thread migration:Distributed EquinoxDynamic scaling for the cloudVertical scaling of running applications
3 Improvements for migration agents:Migrate locked objectsMigrate interlocked threadsMigratable Resources: Sockets, Files
[email protected] Sovereign March 22, 2011 32 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook
Thank you for listening!
Questions?
[email protected] Sovereign March 22, 2011 33 / 34
eclipsesource-logo
Strategies Recording Modified VM Outlook
References
http://www.so.in.tum.de/sovereign
http://www.terracotta.org
http://www.eclipse.org/aspectj
http://wikis.sun.com/display/MaxineVM
Quitadamo, Mobile JikesRVM: A framework to supporttransparent Java thread migration, 2008
[email protected] Sovereign March 22, 2011 34 / 34
eclipsesource-logo
TraceRAPWidgetCreation.aspectj
1 Object around() : traceRAPWidgetCreation() {2 if (primary) {3 widget = proceed();4 // record object5 recordObject(widget);6 return widget;7 } else {8 // replay object9 widget = replayObject();
10 return widget;11 }12 }
[email protected] Sovereign March 22, 2011 1 / 2
eclipsesource-logo
TraceRAPMethodCalls.aspectj
1 Object around() : traceRAPMethodCalls() {2 if (primary) {3 // normal operation4 return proceed();5 } else {6 // do nothing7 return null;8 }9 }
[email protected] Sovereign March 22, 2011 2 / 2