taming java agents
DESCRIPTION
A Java agents are pluggable self contained components that run embedded in a JVM and intercept the classloading process. They were introduced in Java 5 along with the powerful java.lang.instrument package. Java agents can be loaded statically at startup or dynamically (programmatically) at runtime to attach to a running process in a fail-safe fashion. Java agents were an awesome addition to the JVM as it opened a lot of opportunities for tool designers and changed Java tooling landscape quite drastically. In conjunction with Java bytecode manipulation libraries it is now possible to do amazing things to Java classes: we can experiment with programming models, redefine classes in runtime, record execution flow, etc. The presentation provides an overview of Java agents’ functionality along with the usage examples and real world experiences. You will learn, how to implement an agent and apply Instrumentation API in combination with bytecode manipulation libraries to solve interesting tasks.TRANSCRIPT
Taming Java Agents
Anton Arhipov ZeroTurnaround, JRebel @antonarhipov
-‐javaagent
java.lang.instrument
java.lang.instrument import java.lang.instrument.ClassFileTransformer; import java.lang.instrument.InstrumentaBon; public class Agent { public staBc void premain(String args, InstrumentaBon inst) { inst.addTransformer(new ClassFileTransformer { … }); } public staBc void agentmain(String args, InstrumentaBon inst) { premain(args, inst); } }
META-INF/MANIFEST.MF Premain-Class: Agent Agent-Class: Agent
java –javaagent:agent.jar …
META-‐INF/MANIFEST.MF
• Premain-‐Class • Agent-‐Main • Boot-‐Class-‐Path • Can-‐Redefine-‐Classes • Can-‐Retransform-‐Classes
j.l.instrument.ClassFileTransformer byte[] transform(ClassLoader loader, String className, Class<?> classBeingRedefined, ProtecBonDomain protecBonDomain, byte[] classfileBuffer);
public class MyTransformer implements ClassFileTransformer { public void byte[] transform(ClassLoader loader, String className, Class<?> classBeingRedefined, ProtecBonDomain protecBonDomain, byte[] classfileBuffer){ ClassPool cp = ClassPool.getDefault(); CtClass ct = cp.makeClass( new ByteArrayInputStream(classfileBuffer)); … … … return ct.toBytecode();
public class MyTransformer implements ClassFileTransformer { public void byte[] transform(ClassLoader loader, String className, Class<?> classBeingRedefined, ProtecBonDomain protecBonDomain, byte[] classfileBuffer){ ClassPool cp = ClassPool.getDefault(); CtClass ct = cp.makeClass( new ByteArrayInputStream(classfileBuffer)); … … … return ct.toBytecode();
Enter Javassist!
public class MyTransformer implements ClassFileTransformer { public void byte[] transform(ClassLoader loader, String className, Class<?> classBeingRedefined, ProtecBonDomain protecBonDomain, byte[] classfileBuffer){
ct.addMethod(CtNewMethod.make( "public void foo() {} ", ct));
public class MyTransformer implements ClassFileTransformer { public void byte[] transform(ClassLoader loader, String className, Class<?> classBeingRedefined, ProtecBonDomain protecBonDomain, byte[] classfileBuffer){
CtMethod m = ct.getMethod("foo", "()V"); m.insertA_er("System.out.println(\"..\")");
void addTransformer(ClassFileTransformer transformer, boolean canRetransform); void appendToBootstrapClassLoaderSearch(JarFile jarfile); void appendToSystemClassLoaderSearch(JarFile jarfile); Class[] getAllLoadedClasses(); Class[] getIniBatedClasses(ClassLoader loader); void redefineClasses(ClassDefiniBon... classes); void retransformClasses(Class<?>... classes); . . .
j.l.instrument.InstrumentaBon
Example 1: Simple Trace public class Main { public staBc void main(String[] args) { for (int i = 0; i < args.length; i++) System.out.println(args[i]); } } $ java Main 1 2 3 1 2 3
Example 1: Simple Trace
$ java –javaagent:byteman.jar=script:trace.btm Main
RULE trace main entry CLASS Main METHOD main AT ENTRY IF true DO traceln("entering main") ENDRULE
RULE trace main exit CLASS Main METHOD main AT EXIT IF true DO traceln("exiBng main") ENDRULE
entering main 1 2 3 exiBng main
Example 2: Tracing Threads
for (int i = 0; i < args.length; i++) { final String arg = args[i]; Thread thread = new Thread(arg) { public void run() { System.out.println(arg); } }; thread.start(); thread.join(); }
Example 2: Tracing Threads RULE trace thread start CLASS java.lang.Thread METHOD start() IF true DO traceln("*** start for thread " + $0.getName()) ENDRULE
Example 2: Tracing Threads $ java -‐Dorg.jboss.byteman.transform.all \ -‐javaagent:byteman.jar=script:thread.btm,boot:byteman.jar \ Main foo bar baz *** start for thread foo foo *** start for thread bar bar *** start for thread baz baz
Example 3: Side Effects
RULE skip loop iteraBon CLASS Main METHOD main AFTER CALL join IF ($args[$i]).contains("foo") DO $i = $i + 1 ; traceln("skipping iteraBon " + $i) ENDRULE
Example 4: Arach to Process BufferedReader in = new BufferedReader(new InputStreamReader(System.in)); String next = in.readLine(); while (next != null && next.length() > 0 && !next.contains("end")) { final String arg = next; Thread thread = new Thread(arg) { public void run() { System.out.println(arg); } }; thread.start(); thread.join(); next = in.readLine(); }
Example 4: Arach to Process
$ bminstall.sh -‐b –Dorg… <PID> $ bmsubmit.sh script/ex2.btm install rule trace thread start
$ java Main Hello Hello
$ bmsubmit.sh -‐u script/ex2.btm uninstall RULE trace thread start
Test *** start for thread Test Test
Test Test end $
Arach API
VirtualMachine vm = VirtualMachine.arach("2177"); vm.loadAgent("agent.jar", "arg1=x,arg2=y"); vm.detach();
LimitaBon: can’t change class schema
Fault InjecBon VS
Mocking
Example 5: Fault InjecBon @RunWith(BMUnitRunner.class) public class BytemanJUnitTests { @Test(expected=MyServiceUnavailableExcepBon.class) @BMRule(name="throw Bmeout at 1st call", targetClass = "Socket", targetMethod = "connect", acBon = "throw new java.io.IOExcepBon()") public void testErrorInPipeline() throws ExcepBon { // Invokes internally Socket.connect(..): new MyHrpClient("hrp://example.com/data").read(); } }
Example 5: Fault InjecBon @RunWith(BMUnitRunner.class) public class BytemanJUnitTests { @Test(expected=MyServiceUnavailableExcepBon.class) @BMRule(name="throw Bmeout at 1st call", targetClass = "Socket", targetMethod = "connect", acBon = "throw new java.io.IOExcepBon()") public void testErrorInPipeline() throws ExcepBon { // Invokes internally Socket.connect(..): new MyHrpClient("hrp://example.com/data").read(); } }
Example 5: Fault InjecBon @RunWith(BMUnitRunner.class) public class BytemanJUnitTests { @Test(expected=MyServiceUnavailableExcepBon.class) @BMRule(name="throw Bmeout at 1st call", targetClass = "Socket", targetMethod = "connect", acBon = "throw new java.io.IOExcepBon()") public void testErrorInPipeline() throws ExcepBon { // Invokes internally Socket.connect(..): new MyHrpClient("hrp://example.com/data").read(); } }
Example 5: Fault InjecBon @RunWith(BMUnitRunner.class) public class BytemanJUnitTests { @Test(expected=MyServiceUnavailableExcepBon.class) @BMRule(name="throw Bmeout at 1st call", targetClass = "Socket", targetMethod = "connect", acBon = "throw new java.io.IOExcepBon()") public void testErrorInPipeline() throws ExcepBon { // Invokes internally Socket.connect(..): new MyHrpClient("hrp://example.com/data").read(); } }
Example 5: Fault InjecBon @RunWith(BMUnitRunner.class) public class BytemanJUnitTests { @Test(expected=MyServiceUnavailableExcepBon.class) @BMRule(name="throw Bmeout at 1st call", targetClass = "Socket", targetMethod = "connect", acBon = "throw new java.io.IOExcepBon()") public void testErrorInPipeline() throws ExcepBon { // Invokes internally Socket.connect(..): new MyHrpClient("hrp://example.com/data").read(); } }
@Path("/") public String foo() { return "FooBar"; }
@Path("/") public String foo() { return "FooBar"; }
@Path("/") public FooBar foo() { return new FooBar(); }
@Path("/") public String foo() { return "FooBar"; }
@Path("/foobar") public FooBar foo() { return new FooBar(); }
MyObject
MyObject.class
OldClassLoader
Code 101000101 100010010 New code
111000100 101010010
Make changes in IDE
JRebel
Fram
ework
ConfiguraDon (XML, annotaDons,..)
JRebel
java –javaagent:jrebel.jar App
Chronon
• A back-‐in-‐Bme debugger for Java – Recording a program execuBon – Debugging the recording
• What is recorded? – Variable history – Method calls – ExcepBons – Console output – Threads
Recorded data
Memory buffer
ApplicaBon threads
Flusher threads
Recording
java -‐javaagent:recorder.jar=config.txt
Recording Local Variable Changes
• Just a brute-‐force idea: – Instrument *store (astore, istore, etc) instrucBons to write the value to a store
int i = 2; iconst_2 istore_1
iconst_2 istore_1 iload_1 invokestaBc …
Plumbr: Memory leak detecBon
• RunBme applicaBon analysis – vs post-‐mortem heap dump analysis – vs profilers – human operated, rarely usable in producBon
• AdapBve introspecBon – Monitor high level metrics, cheap to obtain – Collect expensive detailed informaBon when suspects are found
Plumbr
• Memory leak in Java – some objects are created Bme a_er Bme but a reference is forgoren
• Look at when object was created and how long it lives
What is leaking?
Common JDK class instances:
Class instances that handle user interaction:
Instances of a leaking class:
Instances of a class that belong to base application framework:
Time (garbage collection cycles)
-‐javaagent
java.lang.instrument
@antonarhipov [email protected]