relproxy, easy class reload and scripting with java

RelProxy

Easy Class Reload

and Scripting with Java

Jose María Arranz Santamaría January 28, 2015

RelProxy...

● WHAT IS?

● WHY?

● HOW IT WORKS…

● WHERE?

WHAT IS?

What is?

● A simple Java and Groovy automatic class reloader

from sourceo In development … and production if you want

● Converts pure Java on the fastest scripting languageo Compiling on demand, not a new language like BeanShell

o Including an interactive shell

ENVY!!!

Why?

● Envy of Groovy’s “false” scripting

● Groovy is NOT really a scripting/interpreted language, it

“just” compiles on the fly to conventional Java bytecodeo More correctly Groovy is a dynamic language

● I realized this when making an example of embedding

Groovy on ItsNat web framework

Why?

● Envy for automatic class reload of other web

frameworks & dynamic langs with just a page reload

● Auto context reloading (Tomcat etc) is TEDIOUSo Everything is reloaded per file save => slowing all and PermGen

o Session is lost, start again and again

● ItsNat, a Java based web framework, asked his father

for support of automatic class reloading o It suffers of “me too” syndrome

o Very soon I realized it could be an independent project

HOW IT WORKS

How it works

● The Java Reload Proxy

● A shell scripting language named Java

● JSR-223 Java Scripting API

● Bonus: Groovy loves Java. The Groovy Reload Proxy

The Java Reload Proxy


GOAL:

Automatic recompile and reload changed source

files/classes in web with a simple page reload

WITHOUT CONTEXT RELOADING!!

DISABLE IT!! => read the Manual


InvocationHandler

VersionedObject Object

JProxyEngine

Change

Detector And

Compiler

Class

Reloader

java.lang.ref

lect.Proxy

public

method

invoked

ask

reload

needed

reload

JProxyClass

Loader

new

instance

versioned

method

called


● Advantages over JVM HotSwap in Debugo JVM standard HotSwap is limited to change method bodies

RelProxy/JProxy gives you more freedom, even add new classes

● Advantages over Context Reloadingo Context Reloading reloads ANYTHING

RelProxy only loads a subset of your code

o In CR session is lost

o In CR, in practice, reloading happens per file save


● Everything cannot and should not be reloadedo Server state must be kept when source code is changed

o Reloading data hold by singletons is dangerous

● Stateless/functional classes are candidates for

reloadingo In ItsNat, code doing page load/event rendering on request, that is

code dependent from ItsNatServletRequestListener

o The method public void processRequest(ItsNatServletRequest request,ItsNatServletResponse response)

o is called on page load time

The Java Reload Proxypackage example.javaex;

import org.itsnat.core.event.ItsNatServletRequestListener;

import org.itsnat.core.ItsNatServletRequest;

import org.itsnat.core.ItsNatServletResponse;

import org.itsnat.core.html.ItsNatHTMLDocument;

public class JProxyExampleLoadListener implements ItsNatServletRequestListener

{

protected FalseDB db;

public JProxyExampleLoadListener() { }

public JProxyExampleLoadListener(FalseDB db) { this.db = db; }

public void processRequest(ItsNatServletRequest request, ItsNatServletResponse response)

{

System.out.println("JProxyExampleLoadListener 1 " + this.getClass().getClassLoader().hashCode());

new example.javaex.JProxyExampleDocument(request,(ItsNatHTMLDocument)request.getItsNatDocument(),db);

}

}


FalseDB db = new FalseDB();

String pathPrefix = context.getRealPath("/") + "/WEB-INF/jproxyex/pages/";

ItsNatDocumentTemplate docTemplate;

docTemplate = itsNatServlet.registerItsNatDocumentTemplate("jproxyex","text/html",

pathPrefix + "jproxyex.html");

ItsNatServletRequestListener listener = new example.javaex.JProxyExampleLoadListener(db);

docTemplate.addItsNatServletRequestListener(listener);

● In a conventional ItsNat web application, JProxyExampleLoadListener is a singleton

registered with something like this:


public void processRequest(ItsNatServletRequest request, ItsNatServletResponse response)

{

System.out.println("JProxyExampleLoadListener 10 " + this.getClass().getClassLoader().hashCode());

new example.javaex.JProxyExampleDocument(request,(ItsNatHTMLDocument)request.getItsNatDocument(),db);

}

● Remember the JProxyExampleLoadListener code:

● If we are able of reloading JProxyExampleLoadListener we are also able to fully reload

JProxyExampleDocument and dependent code on a page load =>VIEW LOGIC RELOADED

Every page reload

creates a new instance


Because JProxyExampleLoadListener is a singleton

we can reload the code => fields must not change (state)

Q) How can we detect source code changes of the

singleton class and related classes and apply them in real

time?

A) Registering a proxy instead of the original singleton,

when a method is called, it is performed on the proxy

instead of the original Java object => reload classesThis is why com.innowhere.relproxy.jproxy.JProxy exists


package example.javaex;

import java.io.File;

import java.lang.reflect.Method;

import java.util.Arrays;

import java.util.List;

import javax.servlet.*;

import javax.tools.Diagnostic;

import javax.tools.DiagnosticCollector;

import javax.tools.JavaFileObject;

import org.itsnat.core.event.ItsNatServletRequestListener;

import org.itsnat.core.http.HttpServletWrapper;

● Configuration in the servleto Part of this code can be put in a ServletContextListener

import org.itsnat.core.tmpl.ItsNatDocumentTemplate;

import com.innowhere.relproxy.RelProxyOnReloadListener;

import com.innowhere.relproxy.jproxy.*;

public class JProxyExampleServlet extends HttpServletWrapper

{

public JProxyExampleServlet() { }

@Override

public void init(ServletConfig config) throws ServletException

{

super.init(config);

ServletContext context = getServletContext();

String realPath = context.getRealPath("/");

String inputPath = realPath + "/WEB-INF/javaex/code/";

String classFolder = null; // Optional: context.getRealPath("/") + "/WEB-INF/classes";

Iterable<String> compilationOptions = Arrays.asList(new String[]{"-source","1.6","-target","1.6"});

long scanPeriod = 300;

JProxy config params

Java sources to reload

are here!!


RelProxyOnReloadListener proxyListener = new RelProxyOnReloadListener() {

@Override

public void onReload(Object objOld, Object objNew, Object proxy, Method method, Object[] args) {

System.out.println("Reloaded " + objNew + " Calling method: " + method);

}

};

JProxyInputSourceFileExcludedListener excludedListener = new JProxyInputSourceFileExcludedListener() {

@Override

public boolean isExcluded(File file, File rootFolderOfSources) { return false; }

};

JProxyCompilerListener compilerListener = new JProxyCompilerListener(){

@Override

public void beforeCompile(File file) { System.out.println("Before compile: " + file); }

@Override

public void afterCompile(File file) { System.out.println("After compile: " + file); }

};

Listener monitor of class

reloading

Listener monitor of file

compiling

Filter for excluding files

of reloading

The Java Reload ProxyJProxyDiagnosticsListener diagnosticsListener = new JProxyDiagnosticsListener() {

@Override

public void onDiagnostics(DiagnosticCollector<JavaFileObject> diagnostics) {

List<Diagnostic<? extends JavaFileObject>> diagList = diagnostics.getDiagnostics();

int i = 1;

for (Diagnostic<? extends JavaFileObject> diagnostic : diagList) {

System.err.println("Diagnostic " + i);

System.err.println(" code: " + diagnostic.getCode());

System.err.println(" kind: " + diagnostic.getKind());

System.err.println(" line number: " + diagnostic.getLineNumber());

System.err.println(" column number: " + diagnostic.getColumnNumber());

System.err.println(" start position: " + diagnostic.getStartPosition());

System.err.println(" position: " + diagnostic.getPosition());

System.err.println(" end position: " + diagnostic.getEndPosition());

System.err.println(" source: " + diagnostic.getSource());

System.err.println(" message: " + diagnostic.getMessage(null));

i++;

}

}

};


JProxyConfig jpConfig = JProxy.createJProxyConfig();

jpConfig.setEnabled(true)

.setRelProxyOnReloadListener(proxyListener)

.setInputPath(inputPath)

.setJProxyInputSourceFileExcludedListener(excludedListener)

.setScanPeriod(scanPeriod)

.setClassFolder(classFolder)

.setCompilationOptions(compilationOptions)

.setJProxyCompilerListener(compilerListener)

.setJProxyDiagnosticsListener(diagnosticsListener);

JProxy.init(jpConfig);

Configuring JProxy library

Efective configuration


String pathPrefix = context.getRealPath("/") + "/WEB-INF/javaex/pages/";

ItsNatDocumentTemplate docTemplate;

docTemplate = itsNatServlet.registerItsNatDocumentTemplate("javaex","text/html",

pathPrefix + "javaex.html");

FalseDB db = new FalseDB();

ItsNatServletRequestListener listener = JProxy.create(

new example.javaex.JProxyExampleLoadListener(db),ItsNatServletRequestListener.class);

docTemplate.addItsNatServletRequestListener(listener);

}

}

Registers ItsNat HTML

template

Proxy creation wrapping

the load listener singletonRegistering the load listener

singleton associated to the

template


● FalseDB and related are not reloaded in this exampleo It is not proxied and is an external dependency

● Just calling setEnabled(false) in production and

performance impact is ZERO

● JProxy is ItsNat agnostico In spite of previous example is based on ItsNat

● JProxy can be used in similar use cases in your Java

projecto Web or not web

WHO THE FUCK

IS USING

ITSNAT !!!

https://groups.google.com/forum/#!topic/itsnat/CO8Qd-Am3L0

How JProxy can help you

in development time,

a GWT example

YES…

GWT !!

How JProxy can help you in

development time, a GWT example

● We can define normal Java source code folders as

reloadableo No need of source code below WEB-INF/

o No need to publish source code in production

o Of course this is NOT valid for production

● We will show this feature through a GWT-RPC example

● Download and install Eclipse (Luna is supposed) and

Google Plugin for Eclipse

● Create a GWT-RPC projecto Select New/Other/Google/Web Application Project

o Add relproxy-x.y.z.jar to WEB-INF/lib



● Structure generatedo Name/package relproxy_ex_gwt/com.innowhere.relproxyexgwt

relproxy_ex_gwt (root folder of project)

src/com/innowhere/relproxyexgwt

client

GreetingService.java

GreetingServiceAsync.java

Relproxy_ex_gwt.java

server

GreetingServiceImpl.java

shared

FieldVerifier.java

Relproxy_ex_gwt.gwt.xml



● We are only be able to reload classes executed in servero Classes below server/ folder

o That is, the servlet GreetingServiceImpl.java

● Initial code of GreetingServiceImpl

package com.innowhere.relproxyexgwt.server;

import com.google.gwt.user.server.rpc.RemoteServiceServlet;

import com.innowhere.relproxyexgwt.client.GreetingService;

import com.innowhere.relproxyexgwt.shared.FieldVerifier;

/**

* The server side implementation of the RPC service.

*/

@SuppressWarnings("serial")

public class GreetingServiceImpl extends RemoteServiceServlet implements GreetingService {

development time, a GWT examplepublic String greetServer(String input) throws IllegalArgumentException {

// Verify that the input is valid.

if (!FieldVerifier.isValidName(input)) {

// If the input is not valid, throw an IllegalArgumentException back to

// the client.

throw new IllegalArgumentException("Name must be at least 4 characters long");

}

String serverInfo = getServletContext().getServerInfo();

String userAgent = getThreadLocalRequest().getHeader("User-Agent");

// Escape data from the client to avoid cross-site script vulnerabilities.

input = escapeHtml(input);

userAgent = escapeHtml(userAgent);

return "Hello, " + input + "! I am running " + serverInfo +

". It looks like you are using: " + userAgent;

}

/**

* Escape an html string. Escaping data received from the client helps to

* prevent cross-site script vulnerabilities.

*

* @param html the html string to escape

* @return the escaped string

*/

private String escapeHtml(String html) {

if (html == null) {

return null;

}

return html.replaceAll("&", "&").replaceAll("<", "<").replaceAll(">", ">");

}

}

● Converted to use JProxy...




import java.io.File;

import java.lang.reflect.Method;

import java.util.*;

import javax.servlet.*;

import javax.servlet.http.HttpServletRequest;

import javax.tools.*;

import com.google.gwt.user.server.rpc.RemoteServiceServlet;

import com.innowhere.relproxy.RelProxyOnReloadListener;

import com.innowhere.relproxy.jproxy.*;

import com.innowhere.relproxyexgwt.client.GreetingService;

/**

* The server-side implementation of the RPC service.

*/

@SuppressWarnings("serial")

public class GreetingServiceImpl extends RemoteServiceServlet implements

GreetingService {



protected GreetingServiceDelegate delegate;

public void init(ServletConfig config) throws ServletException {

super.init(config);

ServletContext context = config.getServletContext();

String inputPath = context.getRealPath("/") + "/../src/";

JProxyInputSourceFileExcludedListener excludedListener =

new JProxyInputSourceFileExcludedListener() {

@Override

public boolean isExcluded(File file, File rootFolder) {

String absPath = file.getAbsolutePath();

if (file.isDirectory()) return absPath.endsWith(File.separatorChar + "client") ||

absPath.endsWith(File.separatorChar + "shared");

else return absPath.endsWith(GreetingServiceDelegate.class.getSimpleName() + ".java") ||

absPath.endsWith(GreetingServiceImpl.class.getSimpleName() + ".java");

}

};

Your source !!

Folders excluded

Concrete files excluded

String classFolder = null; // Optional: context.getRealPath("/") + "/WEB-INF/classes";

Iterable<String> compilationOptions =

Arrays.asList(new String[]{"-source","1.6","-target","1.6"});

long scanPeriod = 300;

RelProxyOnReloadListener proxyListener = new RelProxyOnReloadListener() {

public void onReload(Object objOld,Object objNew,Object proxy,Method method,Object[] args){

System.out.println("Reloaded " + objNew + " Calling method: " + method);

}

};

JProxyCompilerListener compilerListener = new JProxyCompilerListener(){

public void beforeCompile(File file) {

System.out.println("Before compile: " + file);

}

public void afterCompile(File file) {

System.out.println("After compile: " + file);

}

};

JProxyDiagnosticsListener diagnosticsListener = new JProxyDiagnosticsListener() {

public void onDiagnostics(DiagnosticCollector<JavaFileObject> diagnostics) {

List<Diagnostic<? extends JavaFileObject>> diagList = diagnostics.getDiagnostics();

int i = 1;

for (Diagnostic diagnostic : diagList) {

System.err.println("Diagnostic " + i);

System.err.println(" code: " + diagnostic.getCode());

System.err.println(" kind: " + diagnostic.getKind());

System.err.println(" line number: " + diagnostic.getLineNumber());

System.err.println(" column number: " + diagnostic.getColumnNumber());

System.err.println(" start position: " + diagnostic.getStartPosition());

System.err.println(" position: " + diagnostic.getPosition());

System.err.println(" end position: " + diagnostic.getEndPosition());

System.err.println(" source: " + diagnostic.getSource());

System.err.println(" message: " + diagnostic.getMessage(null));

i++;

}

}

};



JProxyConfig jpConfig = JProxy.createJProxyConfig();

jpConfig.setEnabled(true)

.setRelProxyOnReloadListener(proxyListener)

.setInputPath(inputPath)

.setJProxyInputSourceFileExcludedListener(excludedListener)

.setScanPeriod(scanPeriod)

.setClassFolder(classFolder)

.setCompilationOptions(compilationOptions)

.setJProxyCompilerListener(compilerListener)

.setJProxyDiagnosticsListener(diagnosticsListener);

JProxy.init(jpConfig);

this.delegate = JProxy.create(

new GreetingServiceDelegateImpl(this),GreetingServiceDelegate.class);

} // init



public String greetServer(String input) throws IllegalArgumentException {

try {

return delegate.greetServer(input);

}

catch(IllegalArgumentException ex) {

ex.printStackTrace();

throw ex;

}

catch(Exception ex) {

ex.printStackTrace();

throw new RuntimeException(ex);

}

}

public HttpServletRequest getThreadLocalRequestPublic() {

return getThreadLocalRequest();

}

}



● GreetingServiceImpl is a servlet, therefore a

singleton in practice, we cannot reload this singleton

● This is why we have moved the code to GreetingServiceDelegateImplo implementing the interface GreetingServiceDelegate

● This new delegation singleton is registered on JProxy

and can be reloadedthis.delegate = JProxy.create(

new GreetingServiceDelegateImpl(this),GreetingServiceDelegate.class);



● When a source change is detected JProxy needs to

reload dependent classes in a new class loader ● But the servlet GreetingServiceImpl cannot be

reloaded, neither client and shared classes, nor

GreetingServiceDelegate => EXCLUDED

public boolean isExcluded(File file, File rootFolder) {

String absPath = file.getAbsolutePath();

if (file.isDirectory()) return absPath.endsWith(File.separatorChar + "client") ||

absPath.endsWith(File.separatorChar + "shared");

else return absPath.endsWith(GreetingServiceDelegate.class.getSimpleName() + ".java") ||

absPath.endsWith(GreetingServiceImpl.class.getSimpleName() + ".java");

}



● GreetingServiceDelegateImpl is basically the

same as the original servlet generated


import com.innowhere.relproxyexgwt.shared.FieldVerifier;

public class GreetingServiceDelegateImpl implements GreetingServiceDelegate

{

protected GreetingServiceImpl parent;

public GreetingServiceDelegateImpl() { } // Needed by JProxy

public GreetingServiceDelegateImpl(GreetingServiceImpl parent) {

this.parent = parent;

}

development time, a GWT examplepublic String greetServer(String input) throws IllegalArgumentException {

// Verify that the input is valid.

if (!FieldVerifier.isValidName(input)) {

throw new IllegalArgumentException("Name must be at least 4 characters long");

}

String serverInfo = parent.getServletContext().getServerInfo();

String userAgent = parent.getThreadLocalRequestPublic().getHeader("User-Agent");

input = escapeHtml(input);

userAgent = escapeHtml(userAgent);

return "Hello, " + input + "! I am running " + serverInfo

+ ". It looks like you are using: " + userAgent;

}

private String escapeHtml(String html) {

if (html == null) { return null; }

return html.replaceAll("&", "&").replaceAll("<", "<")

.replaceAll(">", ">");

}

}



● Run this example o Run As / Web Application / GWT Super Dev Mode

● Open http://127.0.0.1:8888/Relproxy_ex_gwt.html

● Click on the “Close” button ● Modify in Eclipse GreetingServiceDelegateImpl

o just change "Hello" by "Hello BROTHER" and save:

return "Hello BROTHER, " + input + "! I am running " + serverInfo

+ ". It looks like you are using: " + userAgent;

● Click again on “Send to Server” button

o The RPC callback will be called

o In this example a page reload is not needed

A shell scripting

language named Java

A shell scripting lang named Java

● Your JDK includes a built-in API for compiling Java fileso Since Java 1.6

o The compiler API used by web app servers to compile servlets and

JSPs when changed (previously generated as servlet)

● RelProxy includes a script named jproxysh to execute

Java from source codeo Supported Windows and Unixes

● To execute code like this: (file example_java_shell)

#!/usr/bin/env jproxysh

String msg = args[0] + args[1];

System.out.println(msg);

System.out.println("example_java_shell 1 ");

example.javashellex.JProxyShellExample.exec();

● Obviously Java code is the code in main method

● jproxysh must be in PATH

● JAVA_HOME and CLASSPATH standard environment

variables must be definedo CLASSPATH must include relproxy-X.Y.jar

● This is the expected hierarchy in this example<root_folder>

example_java_shell (file)

example (folder)

javashellex (folder)

JProxyShellExample.java (file)


● Configuration options are defined by using environment

variables

● Example:

export JAVA_OPTS="-client -Xmx100m"

export JPROXYSH_CACHE_CLASS_FOLDER="/tmp/java_shell_test_classes"

export JPROXYSH_COMPILATION_OPTIONS="-source 1.6 -target 1.6"

./example_java_shell "HELLO " "WORLD!"

Not really needed (just an ex.)

To avoid recompiling (optional)


● Nothings prevents of executing a complete root classo File example_java_shell_complete_class

#!/usr/bin/env jproxysh

import example.javashellex.JProxyShellExample;

public class example_java_shell_complete_class {

public static void main(String[] args) {

String msg = args[0] + args[1];

System.out.println(msg);

System.out.println("example_java_shell_complete_class 1 ");

JProxyShellExample.exec();

}

}


● Of course a conventional root class is valido Yes you can execute a conventional Java program from source code!!

jproxysh $PROJECT/code/example_normal_class.java "HELLO " "WORLD!"

● Or just a code snippet

jproxysh -c 'System.out.print("This code snippet says: ");' \

'System.out.println("Hello World!!");'

● Or just start an interactive shell

jproxysh

JSR-223

Java Scripting API

JSR-223 Java Scripting API

● Yes RelProxy provides an implementation of JSR-223

API for a scripting language named “Java” or “FuckYou”JProxyConfig jpConfig = ...;

JProxyScriptEngineFactory factory = JProxyScriptEngineFactory.create();

ScriptEngineManager manager = new ScriptEngineManager();

manager.registerEngineName("Java", factory);

manager.getBindings().put("msg","HELLO GLOBAL WORLD!");

ScriptEngine engine = manager.getEngineByName("Java");

((JProxyScriptEngine)engine).init(jpConfig);

try

{

Bindings bindings = engine.createBindings();

bindings.put("msg","HELLO ENGINE SCOPE WORLD!");

StringBuilder code = new StringBuilder();

JProxyScriptEngine

has the same API as JProxy + eval(...)

JSR-223 Java Scripting API

code.append( " javax.script.Bindings bindings = context.getBindings(javax.script.ScriptContext.ENGINE_SCOPE); \n");

code.append( " String msg = (String)bindings.get(\"msg\"); \n");

code.append( " System.out.println(msg); \n");

code.append( " bindings = context.getBindings(javax.script.ScriptContext.GLOBAL_SCOPE); \n");

code.append( " msg = (String)bindings.get(\"msg\"); \n");

code.append( " System.out.println(msg); \n");

code.append( " example.javashellex.JProxyShellExample.exec(engine); \n");

code.append( " return \"SUCCESS\";");

String result = (String)engine.eval( code.toString() , bindings);

System.out.println(result);

}

catch(ScriptException ex) { ex.printStackTrace(); }

finally

{

((JProxyScriptEngine)engine).stop(); // Necessary if scanPeriod > 0 was defined

}

The Groovy Reload

Proxy

Err...

Another day...

THANKS &

ENJOY !!https://github.com/jmarranz/relproxy/

https://github.com/jmarranz/relproxy_examples/

Q & A

I’m sorry, half the world uses this stolen image from…(?)

relproxy, easy class reload and scripting with java

Software