sovereign: migrating java threads to improve availability of web applications

eclipsesource-logo

Strategies Recording Modified VM Outlook

Sovereign:Migrating Java Threads

to Improve Availability of Web Applications

Johannes Eickhold, Technical University MunichMarkus Knauer, EclipseSource

Dr. Thomas Fuhrmann, Technical University Munich

March 22, 2011

jeick@so.in.tum.de Sovereign March 22, 2011 1 / 34

eclipsesource-logo

What is high availability for web applications?

Servlet Container

Load Balancer

Servlet Container

Load Balancer

Every node can take overOnly works if app is stateless

eclipsesource-logo

What is RAP?

RAP supportsReusing RCP’s UI thread→ single sourcingJob framework

eclipsesource-logo

Challenge: Transfer Server Side State

Servlet Container

Load Balancer

sticky sessions!no-fail-over = true

GoalEnable failsave operation of all RAP features

eclipsesource-logo

Overview

1 Strategies

2 RecordingStrategy ExplainedExampleDemo

3 Modified VMStrategy ExplainedDemo

4 Outlook

eclipsesource-logo

Overview

1 Strategies

4 Outlook

eclipsesource-logo

Two Strategies for Enabling Session Failover

1 Recording StrategyRecords the widget state to shared heapOperates on application level

2 Modified VM StrategyEnables thread migration on the JVM level

eclipsesource-logo

Recording Strategy

eclipsesource-logo

Modified VM Strategy

replicate threads

primary VM secondary VM

freeze thread + replicate state and data + continue thread =thread migration

eclipsesource-logo

Recording Strategy vs. Modified VM Strategy

record replay

UIThread UIThread

byte code weaving unmodified applicationunmodified Hot Spot VM enhanced Maxine VMTerracotta shared heap object and thread replication

eclipsesource-logo

Strategies Recording Modified VM Outlook Strategy Explained Example Demo

Overview

1 Strategies

4 Outlook

eclipsesource-logo

Recording Strategy

Modifications to existing RAP framework1 Record widget creation order on primary2 Replay recorded widgets during creation phase on

secondary3 Cluster the session including widget tree4 Remove non-serializable UI thread from session

Incrementally add failover support for additional RAP widgets

eclipsesource-logo

Lifecycle of a RAP application

Shared Heap

Client

UI Thread

request

Initial request creates the UI thread

eclipsesource-logo

Shared Heap

Client

UI Thread

request

UI thread executes createUI() and creates widget graph

eclipsesource-logo

record

Shared Heap

Client

UI Thread

request

The widget graph is stored on the shared heap

eclipsesource-logo

response

Shared Heap

Client

UI Thread

Servlet responds with rendered UI to client.

eclipsesource-logo

Hot Standby Takes Over

Shared Heap

Client

FAILURE

UI Thread

request

Primary servlet container fails and secondary jumps in

eclipsesource-logo

UI Thread Shared Heap

Client

request

UI Thread

FAILURE

Primary servlet container fails and secondary jumps in

eclipsesource-logo

Client

request

UI Thread

FAILURE

Skip widget creation in new UI thread!

eclipsesource-logo

Failover Situation

replay

Client

request

UI Thread

FAILURE

Widgets are restored from the shared heap

eclipsesource-logo

Failover Situation

response

Client

UI Thread

FAILURE

We achieved transparent failover!

eclipsesource-logo

Implementation

You could implement this yourselfor

reuse existing technologies.

AspectJ is used to:Record order in which widgets are createdTrack method calls to enable method skipping

Terracotta provides a shared heap

eclipsesource-logo

Recap of Recording Strategy

ShareTIMs: Equinox,JettyConfigure RAP(tc-config.xml)

RecordTracing widgetcreation withAspectJSkip methodson secondary

No silver bullet!RAP application has to be designed for failover

eclipsesource-logo

Jump to the example

Example RAP application: "swt layout"Aspects implementing the recording/replay

eclipsesource-logo

SWT Layout Example

1 Label l = new Label(shell, SWT.NONE);2 l.setText("relocate me!");3 ...4 Button b = new Button(shell, SWT.PUSH);5 b.setText("left");6 b.addSelectionListener(new SelectionAdapter() {7 public void widgetSelected(SelectionEvent e) {8 localCounter ++;9 globalCounter++;

10 l.setText("relocated "11 + localCounter + "/" + globalCounter12 + " times");13 }14 });

eclipsesource-logo

Trace RAP Widget Creation and Method Calls

UIThread Button shared heap

setText()

update

On primary

UIThread Button shared heap

restore

setText()

On secondary

eclipsesource-logo

Recording Demo

eclipsesource-logo

Strategies Recording Modified VM Outlook Strategy Explained Demo

Overview

1 Strategies

4 Outlook

eclipsesource-logo

Modified VM Strategy

Extensions to the Maxine research VMObject replicationThread replication

All VMs in web cluster execute:

MigrationAgent + target application

eclipsesource-logo

Maxine VM Internals

Maxine VM is written in Java (metacircular)

ThreadsUses pthreadsOne-on-one mapping: Java↔ pthreadSafepoint operations inserted at backward branchesFreezing of threads only via VmOperation

CompilersBaseline: template JITOptimizing: port of C1X from HotSpot or CPS

Details in wiki: http://wikis.sun.com/display/MaxineVM

eclipsesource-logo

Migration via Agents

RAP clusteringAgents run next to Equinox in same VMReplication only used for UI threads of RAP sessions

migrationAgent.migrate(uiThread);

eclipsesource-logo

Continuous Thread Replication

On primary:

Client RequestThread Replicator UIThread Secondary

HTTP request

handle(request)

notify(request)

handle(request)

finished

replicate(UIThread)

frames & objects

HTTP response

eclipsesource-logo

Replication of Stack Frames

On secondary:

Thread

receivingmigration

VmThread.run()

ThreadStub.run()

nativeThreadCreate()

receivesreplicated thread

preparewait

stack of ThreadStub

eclipsesource-logo

On secondary:

receivesdebug information

receivingmigration

agentcreate

& rewire

stack buffer

MigratedThread.run()

method1()

method2()

methodN()

frame n frame 2 frame 1...

eclipsesource-logo

On secondary:

receivingmigration

stack of ThreadStub

resize

VmThread.run()

ThreadStub.run()

eclipsesource-logo

On secondary:

receivingmigration

stack of ThreadStub

resize

VmThread.run()

ThreadStub.run()

eclipsesource-logo

On secondary:

method2()

method1()

methodN()

receivingmigration

agent bufferinsert

stack of ThreadStub

VmThread.run()

ThreadStub.run()

eclipsesource-logo

On secondary:

ThreadStub.patch()

methodN()

method2()

method1()

receivingmigration

agent rewire

stack of ThreadStub

VmThread.run()

ThreadStub.run()

eclipsesource-logo

Inspecting the Migration Algorithm

RCP based tooling:Inspect migrated entities at VM level

eclipsesource-logo

Migration Demo

eclipsesource-logo

Overview

1 Strategies

4 Outlook

eclipsesource-logo

Conclusion

Clustering RAP’s UIThreads is possible

Achieves transparent failover

eclipsesource-logo

Future Work

1 Future directions for clustered RAP:Support more SWT widgetsJFace and WorkbenchSupport alternatives to Terracotta

2 Potential for Java VMs with thread migration:Distributed EquinoxDynamic scaling for the cloudVertical scaling of running applications

3 Improvements for migration agents:Migrate locked objectsMigrate interlocked threadsMigratable Resources: Sockets, Files

eclipsesource-logo

Thank you for listening!

Questions?

eclipsesource-logo

References

http://www.so.in.tum.de/sovereign

http://www.terracotta.org

http://www.eclipse.org/aspectj

http://wikis.sun.com/display/MaxineVM

Quitadamo, Mobile JikesRVM: A framework to supporttransparent Java thread migration, 2008

eclipsesource-logo

TraceRAPWidgetCreation.aspectj

1 Object around() : traceRAPWidgetCreation() {2 if (primary) {3 widget = proceed();4 // record object5 recordObject(widget);6 return widget;7 } else {8 // replay object9 widget = replayObject();

10 return widget;11 }12 }

eclipsesource-logo

TraceRAPMethodCalls.aspectj

1 Object around() : traceRAPMethodCalls() {2 if (primary) {3 // normal operation4 return proceed();5 } else {6 // do nothing7 return null;8 }9 }

sovereign: migrating java threads to improve availability of web applications

Technology

threads -...

oecd sovereign borrowing outlook 2017 sovereign …the oecd...

programming with threads in unix - uni klagenfurt |...

sovereign god, sovereign state, sovereign self

threads chair’s message research threads

migrating soa

the sovereign grant and sovereign grant reserve

severed threads - excerpt from threads, book 1

1 threads & scheduling what are threads? vs. processes where...

viscose embroidery threads & rayon embroidery threads

migrating powerbuilder applications · 2020. 5. 20. · 9...

1 chapter 5 threads 2 contents overview benefits user and...

industrial threads by threads technology (india), mumbai

type of threads - how to identify threads

migrating westward

threads. threads overview multithreading models threading...

taming java programming language threads...

unix threads operating systems. user thread packages pthread...

migrating to enterprise extender ?migrating to enterprise...

migrating trade