Java Style, Hot Deployment, Supporter behind JVM Java Agent

Posted by genistaff on Wed, 11 Sep 2019 03:40:34 +0200

We don't really have many opportunities to write Java Agents in general, or we hardly need them.But in fact, we've been using it all along, and there's a lot of contact.These techniques all use Java Agent technology, so you can see why.

- Debugging capabilities of individual Java IDE s, such as eclipse, IntelliJ;

- Hot deployment features, such as JRebel, XRebel, spring-loaded;

- various online diagnostic tools, such as Btrace, Greys, and Ali's Arthas;

- Various performance analysis tools, such as Visual VM, JConsole, etc.

Java Agent is translated literally as Java Agent, and there is another name called call as Java Probe.First of all, Java Agent is a jar package, but this jar package can not run independently, it needs to be attached to our target JVM process.Let's understand these two terms.

Proxy: For example, we need to know some running metrics of the target JVM, which we can achieve through Java Agent. So it looks like a proxy effect. The last indicator we get is the target JVM, but we get it through Java Agent, which is like a proxy for the target JVM;

Probe: I feel very image of this statement, once the JVM runs, it is a black box for the outside world.The Java Agent can plug into the JVM like a pin, discover what we want, and inject it.

Take the above examples of techniques that we would normally use.Take the IDEA debugger for example. When debugging is turned on, you can see the structure and content of the current context variable in the debugger panel. You can also run some simple code in the watches panel, such as assigning values.There are also tools for troubleshooting online, such as Btrace and Arthas. For example, there are interfaces that do not return results as expected, but there are no errors in the log. In this case, we can only find out the package name, class name, method name, etc. of the method, without modifying the deployment service, we can find the parameters, return values, exceptions and other information of the call.

The above is just about probing, and hot deployment is more than just probing.Hot deployment means that the latest code logic is guaranteed to work on the service without restarting it.When we modify a class, we replace the previous byte code with the byte code corresponding to the new code through Java Agent's instrument mechanism.

Java Agent Structure


The Java Agent eventually exists as a jar package.There are two main parts, one is the implementation code and the other is the configuration file.

The configuration file is placed in the META-INF directory and is named MANIFEST.MF.Includes the following configuration items:

Manifest-Version: Version number
Created-By:Creator
Agent-Class: Class of the agentmain method
Can-Redefine-Classes: Can classes be redefined
Can-Retransform-Classes: Is byte code substitution possible
Premain-Class: The class where the premain method is located

The entry class implements both the agent main and premain methods, and what functionality the method implements is determined by your needs.

Java Agent implementation and use

The next step is to implement a simple Java Agent based on Java 1.8, which mainly implements two simple functions:

1. Print the names of all currently loaded classes;

2. Monitor a specific method by dynamically inserting simple code into it and getting method return values;

Byte code modification technology is mainly used in inserting code into the method. Byte code modification technology mainly includes javassist, ASM, and advanced encapsulation of ASM extensible cglib. This example uses javassist.So you need to introduce related maven packages.

<dependency>
    <groupId>javassist</groupId>
    <artifactId>javassist</artifactId>
    <version>3.12.1.GA</version>
</dependency>

Implementing entry classes and functional logic

As mentioned above in the entry class, two methods, agentmain and premain, are implemented.The runtime of these two methods is different.From the way Java Agent is used, Java Agent has two ways to start, one is to start with the JVM in the form of JVM startup parameter - javaagent:xxx.jar, in which case the premain method is called and executed before the main method of the main process.Another is to attach dynamically to the target JVM using the loadAgent method, in which case the agent main method is executed.

Loading Method Execution Method
-javaagent:xxx.jar parameter form premain
Dynamic attach agentmain

The code is implemented as follows:

package kite.lab.custom.agent;

import java.lang.instrument.Instrumentation;

public class MyCustomAgent {
    /**
     * jvm Start as a parameter, run this method
     * @param agentArgs
     * @param inst
     */
    public static void premain(String agentArgs, Instrumentation inst){
        System.out.println("premain");
        customLogic(inst);
    }

    /**
     * Start with dynamic attach, run this method
     * @param agentArgs
     * @param inst
     */
    public static void agentmain(String agentArgs, Instrumentation inst){
        System.out.println("agentmain");
        customLogic(inst);
    }

    /**
     * Print all loaded class names
     * Modify Byte Code
     * @param inst
     */
    private static void customLogic(Instrumentation inst){
        inst.addTransformer(new MyTransformer(), true);
        Class[] classes = inst.getAllLoadedClasses();
        for(Class cls :classes){
            System.out.println(cls.getName());
        }
    }
}

We see that both methods have parameters, agentArgs and inst, which are parameters that we bring in when we start Java Agent s, such as -javaagent:xxx.jar agentArgs.Instrumentation Java is an open implementation for byte code modification and program monitoring.The printing of loaded classes and modified byte codes that we want to implement is based on it.One of these methods, inst.getAllLoadedClasses(), implements the ability to get the classes that are loaded as a result.

The inst.addTransformer method is the key to the byte code modification. The next parameter is the implementation class that implements the byte code modification. The code is as follows:

public class MyTransformer implements ClassFileTransformer {

    @Override
    public byte[] transform(ClassLoader loader, String className, Class<?> classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) throws IllegalClassFormatException {
        System.out.println("Loading classes:"+ className);
        if (!"kite/attachapi/Person".equals(className)){
            return classfileBuffer;
        }

        CtClass cl = null;
        try {
            ClassPool classPool = ClassPool.getDefault();
            cl = classPool.makeClass(new ByteArrayInputStream(classfileBuffer));
            CtMethod ctMethod = cl.getDeclaredMethod("test");
            System.out.println("Get method name:"+ ctMethod.getName());

            ctMethod.insertBefore("System.out.println(\" Dynamically Inserted Print Statement \");");
            ctMethod.insertAfter("System.out.println($_);");

            byte[] transformed = cl.toBytecode();
            return transformed;
        }catch (Exception e){
            e.printStackTrace();

        }
        return classfileBuffer;
    }
}

The logic of the code above is to insert a print statement at the beginning of the test method when the loaded class is kite.attachapi.Person. The print content is "dynamically inserted print statement". At the end of the test method, the return value is printed, where $_is the return value, which is the specific identifier in javassist..

MANIFEST.MF Profile

Create a file named MANIFEST.MF under the directory resources/META-INF/with the following configuration:

Manifest-Version: 1.0
Created-By: fengzheng
Agent-Class: kite.lab.custom.agent.MyCustomAgent
Can-Redefine-Classes: true
Can-Retransform-Classes: true
Premain-Class: kite.lab.custom.agent.MyCustomAgent

Configure pom settings for packaging

Finally, the Java Agent exists as a jar package, so the last step is to type the above into a jar package.

Add the following configuration to the pom file

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-assembly-plugin</artifactId>
            <configuration>
                <archive>
                    <manifestFile>src/main/resources/META-INF/MANIFEST.MF</manifestFile>
                </archive>
                <descriptorRefs>
                    <descriptorRef>jar-with-dependencies</descriptorRef>
                </descriptorRefs>
            </configuration>
        </plugin>
    </plugins>
</build>

Using maven's maven-assembly-plugin plug-in, note that you want to specify the path of MANIFEST.MF with manifestFile, then jar-with-dependencies to type in the dependent package.

This is a packaging method, which requires a separate MANIFEST.MF cooperation, and another way, which does not require a separate MANIFEST.MF configuration file to be added to the project, but is completely configurable in the pom file.

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-assembly-plugin</artifactId>
            <executions>
                <execution>
                    <goals>
                        <goal>attached</goal>
                    </goals>
                    <phase>package</phase>
                    <configuration>
                        <descriptorRefs>
                            <descriptorRef>jar-with-dependencies</descriptorRef>
                        </descriptorRefs>
                        <archive>
                            <manifestEntries>
                                <Premain-Class>kite.agent.vmargsmethod.MyAgent</Premain-Class>
                                <Agent-Class>kite.agent.vmargsmethod.MyAgent</Agent-Class>
                                <Can-Redefine-Classes>true</Can-Redefine-Classes>
                                <Can-Retransform-Classes>true</Can-Retransform-Classes>
                            </manifestEntries>
                        </archive>
                    </configuration>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

This way, the content of MANIFEST.MF is fully written in the pom configuration, and the configuration information is automatically generated into the MANIFEST.MF configuration file when packaged.

Run Packaging Command

Now it's easy to execute a maven command.

mvn assembly:assembly

The final typed jar package is generated to the target directory by default in the format "project name-version number-jar-with-dependencies.jar".

Run packaged Java Agent

Write a simple test project to target the JVM and later hook the Java Agent onto the test project in two ways.

package kite.attachapi;

import java.util.Scanner;

public class RunJvm {

    public static void main(String[] args){
        System.out.println("Call test method by number key 1");
        while (true) {
            Scanner reader = new Scanner(System.in);
            int number = reader.nextInt();
            if(number==1){
                Person person = new Person();
                person.test();
            }
        }
    }
}

There is only one simple main method above, which guarantees that the thread does not exit by while, and calls the person.test() method when the number 1 is entered.

The following is the Person class

package kite.attachapi;

public class Person {

    public String test(){
        System.out.println("Execute test methods");
        return "I'm ok";
    }
}

Run as command line

Because the project was created in IDEA, I added parameters directly to IDEA's "Run/Debug Configurations" to save time.

-javaagent:/java-agent Route/lab-custom-agent-1.0-SNAPSHOT-jar-with-dependencies.jar

You can then run directly to see the effect and see the loaded class name.Then enter the numeric key "1" and you will see the modified byte code.

Run as dynamic attach

Before testing, run the test item and remove the previous parameters.After running, find the process id of this one, usually using jps-l.

Dynamic attach ments are implemented in code as follows:

public class AttachAgent {

    public static void main(String[] args) throws Exception{
        VirtualMachine vm = VirtualMachine.attach("pid(Process Number)");
        vm.loadAgent("java-agent Route/lab-custom-agent-1.0-SNAPSHOT-jar-with-dependencies.jar");
    }
}

Running the main method above and typing "1" in the test program yields the same results as the above figure.

Did you find that the simple functions we've implemented here are a bit like BTrace and Arrhhas?We intercepted a specified method, inserted code into it, and got the result back.If we turn the method name into a configurable item and save the results back to a public location, such as a memory database, would we be able to detect online problems as easily as Arthas?Arthas is much more complex, of course, but the principles are the same.

Implementation of sun.management.Agent

I don't know if you've ever used tools like visual VM or JConsole. In fact, they are implemented using management-agent.jar, a Java Agent.If we want Java services to allow remote viewing of JVM information, these parameters are often configured as follows:

-Dcom.sun.management.jmxremote
-Djava.rmi.server.hostname=192.168.1.1
-Dcom.sun.management.jmxremote.port=9999
-Dcom.sun.management.jmxremote.rmi.port=9999
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false

These parameters are defined by management-agent.jar.

When we get into the management-agent.jar package, we see that there is only one MANIFEST.MF configuration file, which is:

Manifest-Version: 1.0
Created-By: 1.7.0_07 (Oracle Corporation)
Agent-Class: sun.management.Agent
Premain-Class: sun.management.Agent

You can see that the entry class is sun.management.Agent, into which you can find the agentmain and premain and see their logic.At the beginning of this class, you can see the parameter definitions that we need to turn on to turn on remote JVM monitoring for services.

Don't stint on your "recommendation"

Welcome to update this series and other articles from time to time
Ancient kites, enter the public number to join the communication group

Topics: Java jvm Maven Apache