Self taught JVM 1. Loading mechanism of JVM classes from JDK source code board

Posted by iantresman on Wed, 22 Dec 2021 02:35:35 +0100

1, General flow of java command execution code:

The loadClass class loading process includes the following steps:

Load = > verify = > prepare = > parse = > initialize = > use = > unload

Load: find and read bytecode files on the hard disk through IO. The class used will be loaded. In the loading phase, a Java. Net file representing this class will be generated in memory Lang. class object, as the access entry of this class in various data (the entry of the method area)

Verification: verify the correctness of bytecode file

Preparation: allocate memory to static variables of the class and assign default values

Resolve: replaces symbol references with direct references. In this stage, some static methods are replaced with pointers or handles to the actual storage memory of the data (that is, symbolic references are replaced with direct references). This is the process of static linking (done during loading). Dynamic link is the process of replacing symbolic reference with direct reference during program running.

Initialization: initialize the static variable of the class to the specified value and execute the static code block.

                

After the class is loaded into the method area, it mainly includes: runtime constant pool, type information, field information, method information, reference of class loader, reference of corresponding class instance, etc.

1) class loader reference: the reference from this class to the class loader instance

2) reference of corresponding class instance: after loading class information into the method area, the class loader creates an object instance of corresponding class type and puts it into the heap as the entry and entry point for the program to access the class definition in the method area.

Note: the JVM loads classes on demand. If the main class needs to use other classes during operation, these classes will be loaded step by step.

public class ClassLoaderTest {

    @Test
    public void loadClass01(){
        new A();
        System.out.println("*************load test************");
        B b = null;
    }
}

class A{
    static{
        System.out.println("this is class A");
    }

    public A() {
        System.out.println("initialize A");
    }
}
class B{
    static{
        System.out.println("this is class B");
    }

    public B() {
        System.out.println("initialize B");
    }
}


+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
The results are as follows:
this is class A
initialize A
*************load test************

Although the variable of B is created, it is not instantiated, and only a class will be created and put into memory.

2, Classloader and parental delegation mechanism

Class loading is mainly realized through class loader. Class loader in JAVA:

1) bootstrap classloader: it is responsible for loading the core class library under the lib directory of JRE that supports the operation of the JVM. Such as DT jar

2) extension class loader (ExtClassLoader): it is responsible for loading the Jar package in the ext extension directory under the lib directory of JRE that supports the operation of the JVM

3) application class loader (AppClassLoader): it is responsible for loading the class package under the ClassPath path, mainly loading the business and technical code in the project.

4) custom class loader: load the class package under the path specified by the user

public class ClassLoaderTest {

    @Test
    public void classLoader01(){
        System.out.println("String:" + String.class.getClassLoader());
        System.out.println("DESKeyFactory:" + DESKeyFactory.class.getClassLoader());
        System.out.println("ClassLoaderTest:" + ClassLoaderTest.class.getClassLoader());
        System.out.println("************************************************************************");

        ClassLoader appClassLoader = ClassLoader.getSystemClassLoader();
        ClassLoader extClassLoader = appClassLoader.getParent();
        ClassLoader bootstrapClassLoader = extClassLoader.getParent();
        System.out.println("bootstrapLoader:" + bootstrapClassLoader);
        System.out.println("extClassLoader" + extClassLoader);
        System.out.println("appClassLoader:" + appClassLoader);
        System.out.println("************************************************************************");

        URL[] urLs = Launcher.getBootstrapClassPath().getURLs();
        for(URL url : urLs){
            System.out.println(url);
        }
        System.out.println("************************************************************************");

        System.out.println("extClassLoader load file:" + System.getProperty("java.ext.dirs"));
        System.out.println("************************************************************************");

        System.out.println("appClassLoader load file:" + System.getProperty("java.class.path"));
    }
}



+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Execution results:
String:null
DESKeyFactory:sun.misc.Launcher$ExtClassLoader@6d03e736
ClassLoaderTest:sun.misc.Launcher$AppClassLoader@18b4aac2
************************************************************************
bootstrapLoader:null
extClassLoadersun.misc.Launcher$ExtClassLoader@6d03e736
appClassLoader:sun.misc.Launcher$AppClassLoader@18b4aac2
************************************************************************
file:/D:/a_install/jdk1.8.0_162/jre/lib/resources.jar
file:/D:/a_install/jdk1.8.0_162/jre/lib/rt.jar
file:/D:/a_install/jdk1.8.0_162/jre/lib/sunrsasign.jar
file:/D:/a_install/jdk1.8.0_162/jre/lib/jsse.jar
file:/D:/a_install/jdk1.8.0_162/jre/lib/jce.jar
file:/D:/a_install/jdk1.8.0_162/jre/lib/charsets.jar
file:/D:/a_install/jdk1.8.0_162/jre/lib/jfr.jar
file:/D:/a_install/jdk1.8.0_162/jre/classes
************************************************************************
extClassLoader load file:D:\a_install\jdk1.8.0_162\jre\lib\ext;C:\Windows\Sun\Java\lib\ext
************************************************************************
appClassLoader load file:D:\a_install\idea\IntelliJ IDEA 2019.3.3\lib\idea_rt.jar;D:\a_install\idea\IntelliJ IDEA 2019.3.3\plugins\junit\lib\junit5-rt.jar;D:\a_install\idea\IntelliJ IDEA 2019.3.3\plugins\junit\lib\junit-rt.jar;D:\a_install\jdk1.8.0_162\jre\lib\charsets.jar;D:\a_install\jdk1.8.0_162\jre\lib\deploy.jar;D:\a_install\jdk1.8.0_162\jre\lib\ext\access-bridge-64.jar;D:\a_install\jdk1.8.0_162\jre\lib\ext\cldrdata.jar;D:\a_install\jdk1.8.0_162\jre\lib\ext\dnsns.jar;D:\a_install\jdk1.8.0_162\jre\lib\ext\jaccess.jar;D:\a_install\jdk1.8.0_162\jre\lib\ext\jfxrt.jar;D:\a_install\jdk1.8.0_162\jre\lib\ext\localedata.jar;D:\a_install\jdk1.8.0_162\jre\lib\ext\nashorn.jar;D:\a_install\jdk1.8.0_162\jre\lib\ext\sunec.jar;D:\a_install\jdk1.8.0_162\jre\lib\ext\sunjce_provider.jar;D:\a_install\jdk1.8.0_162\jre\lib\ext\sunmscapi.jar;D:\a_install\jdk1.8.0_162\jre\lib\ext\sunpkcs11.jar;D:\a_install\jdk1.8.0_162\jre\lib\ext\zipfs.jar;D:\a_install\jdk1.8.0_162\jre\lib\javaws.jar;D:\a_install\jdk1.8.0_162\jre\lib\jce.jar;D:\a_install\jdk1.8.0_162\jre\lib\jfr.jar;D:\a_install\jdk1.8.0_162\jre\lib\jfxswt.jar;D:\a_install\jdk1.8.0_162\jre\lib\jsse.jar;D:\a_install\jdk1.8.0_162\jre\lib\management-agent.jar;D:\a_install\jdk1.8.0_162\jre\lib\plugin.jar;D:\a_install\jdk1.8.0_162\jre\lib\resources.jar;D:\a_install\jdk1.8.0_162\jre\lib\rt.jar;D:\projects\extraTools\FileProcessor\target\test-classes;D:\projects\extraTools\FileProcessor\target\classes;D:\a_install\maven\repository\com\alibaba\fastjson\1.2.47\fastjson-1.2.47.jar;D:\a_install\maven\repository\org\apache\poi\poi\4.0.1\poi-4.0.1.jar;D:\a_install\maven\repository\commons-codec\commons-codec\1.11\commons-codec-1.11.jar;D:\a_install\maven\repository\org\apache\commons\commons-collections4\4.2\commons-collections4-4.2.jar;D:\a_install\maven\repository\org\apache\commons\commons-math3\3.6.1\commons-math3-3.6.1.jar;D:\a_install\maven\repository\org\apache\poi\poi-ooxml\4.0.1\poi-ooxml-4.0.1.jar;D:\a_install\maven\repository\org\apache\poi\poi-ooxml-schemas\4.0.1\poi-ooxml-schemas-4.0.1.jar;D:\a_install\maven\repository\org\apache\xmlbeans\xmlbeans\3.0.2\xmlbeans-3.0.2.jar;D:\a_install\maven\repository\org\apache\commons\commons-compress\1.18\commons-compress-1.18.jar;D:\a_install\maven\repository\com\github\virtuald\curvesapi\1.05\curvesapi-1.05.jar;D:\a_install\maven\repository\dom4j\dom4j\1.6.1\dom4j-1.6.1.jar;D:\a_install\maven\repository\xml-apis\xml-apis\1.0.b2\xml-apis-1.0.b2.jar;D:\a_install\maven\repository\junit\junit\4.12\junit-4.12.jar;D:\a_install\maven\repository\org\hamcrest\hamcrest-core\1.3\hamcrest-core-1.3.jar;D:\a_install\maven\repository\com\ibm\db2\jcc\db2-spring-framework\0.0.1\db2-spring-framework-0.0.1.jar;D:\a_install\maven\repository\com\ibm\db2\jcc\11.5.0.0\jcc-11.5.0.0.jar;D:\a_install\maven\repository\commons-io\commons-io\2.5\commons-io-2.5.jar;D:\a_install\maven\repository\org\freemarker\freemarker\2.3.31\freemarker-2.3.31.jar;D:\a_install\maven\repository\com\ibm\db2\jcc\db2jcc\db2jcc4\db2jcc-db2jcc4.jar;D:\a_install\maven\repository\org\springframework\spring-webmvc\5.2.11.RELEASE\spring-webmvc-5.2.11.RELEASE.jar;D:\a_install\maven\repository\org\springframework\spring-aop\5.2.11.RELEASE\spring-aop-5.2.11.RELEASE.jar;D:\a_install\maven\repository\org\springframework\spring-beans\5.2.11.RELEASE\spring-beans-5.2.11.RELEASE.jar;D:\a_install\maven\repository\org\springframework\spring-context\5.2.11.RELEASE\spring-context-5.2.11.RELEASE.jar;D:\a_install\maven\repository\org\springframework\spring-core\5.2.11.RELEASE\spring-core-5.2.11.RELEASE.jar;D:\a_install\maven\repository\org\springframework\spring-jcl\5.2.11.RELEASE\spring-jcl-5.2.11.RELEASE.jar;D:\a_install\maven\repository\org\springframework\spring-expression\5.2.11.RELEASE\spring-expression-5.2.11.RELEASE.jar;D:\a_install\maven\repository\org\springframework\spring-web\5.2.11.RELEASE\spring-web-5.2.11.RELEASE.jar;D:\a_install\idea\IntelliJ IDEA 2019.3.3\lib\idea_rt.jar

Focus and Description:

1) bootstrap classloader is a class loader for C + +, which is null in java

2) core class library loaded by bootstrap classloader

3) class library of ext extension directory loaded by extClassLoader

4) jar package under classpath directory loaded by appClassLoader

Class loader initialization process:

C + + create JVM initiator instance sun misc. Launcher. sun. misc. The launcher initialization uses the singleton mode to ensure that the virtual machine has only sun misc. Launcher instance.

Two class loaders are created inside the constructor of the Launcher class: ExtClassLoader and AppClassLoader. By default, the JVM uses the instance of the class loader (type: application class loader) returned by the getClassLoader() method of the Launcher class to load the application class

Parental delegation mechanism:

Principle: when loading a class, the current class loader (either an application class loader or a user-defined class loader) will first query whether the class has been loaded in its loaded class. If it has not been loaded, delegate the parent class loader to try to load the class. The parent class loader handles the same logic until the bootstrappclassloader is bootstrapped, If the boot class loader does not find the class in the loaded class, go to the boot class loading path to try to load the class, and then return the loading result to the extension class loader. If the extension class loader determines that the class has been loaded, it returns directly; Otherwise, the extended class loader will go to its own loading path to try to load the class, and then return the loading result to the application class loader. The application class loader will return if it judges that the class has been loaded. If it has not been loaded, it will go to the application class loading path to try to load the class. If it is loaded successfully, it will delete the class, If loading fails, ClassNotFoundException will be reported (if there is a custom class loader, the application class loader will).

Advantages of parental delegation mechanism:

1) water tank security mechanism: prevent the core API class library from being tampered with at will. For example: Java The handwritten version of lang.string class will not be loaded, only Java. String in the core class library will be loaded Lang.string class

2) avoid repeated loading of classes: ensure the uniqueness of the loaded classes. After a class is loaded once, it will not be loaded again.

Overall responsibility entrustment mechanism:
Overall responsibility: when a ClassLoader loads a class, unless another ClassLoader is explicitly used, the class, the class it depends on and the class it references are loaded by this ClassLoader.

Custom class loader:

The custom class loader only needs to inherit Java Lang. classloader class, which has two core methods, loadClass(String,boolean), to implement the parental delegation mechanism; findClass(String), which is empty by default, needs a custom class loader to override the findClass(String) method.

package com.jvm.tt;

import java.io.File;
import java.io.FileInputStream;
import java.lang.reflect.Method;

public class MyClassLoader extends ClassLoader {

    private String classPath;

    public MyClassLoader(String classPath) {
        this.classPath = classPath;
    }

    @Override
    protected Class<?> findClass(String name) throws ClassNotFoundException {
        try {
            byte[] bytes = loadByte(name);
            //defineClass converts a byte array into a class object. This byte array is the final byte array after the class file is read.
            return defineClass(name, bytes, 0, bytes.length);
        } catch (Exception e) {
            throw new ClassNotFoundException();
        }
    }

    private byte[] loadByte(String name) throws Exception{
        name = name.replaceAll("\\.", "/");
        FileInputStream fis = new FileInputStream(new File(classPath + "/" + name + ".class"));
        int available = fis.available();
        byte[] data = new byte[available];
        fis.read(data);
        fis.close();
        return data;
    }


    public static void main(String[] args) throws Exception {
        MyClassLoader myClassLoader = new MyClassLoader("D:\\extra\\keepLearning\\TT\\SpringMVC\\JVMTT\\target\\classes");
        Class<?> clazz = myClassLoader.loadClass("com.jvm.tt.Study");
        Object obj = clazz.newInstance();
        Method main = clazz.getMethod("doStudy");
        main.invoke(obj, null);
        System.out.println(clazz.getClassLoader().getClass().getName());
    }
}

class Study {

    public Study() {
    }

    public void doStudy(){
        System.out.println("studying");
    }
}


+++++++++++++++++++++++++++
Execution results:
studying
sun.misc.Launcher$AppClassLoader

Break the parental delegation mechanism:

package com.jvm.tt;

import java.io.File;
import java.io.FileInputStream;
import java.lang.reflect.Method;

public class NoParentClassLoader extends ClassLoader{

    private String classPath;

    public NoParentClassLoader(String classPath) {
        this.classPath = classPath;
    }

    private byte[] loadByte(String name) throws Exception{
        name = name.replaceAll("\\.", "/");
        FileInputStream fis = new FileInputStream(new File(classPath + "/" + name + ".class"));
        int available = fis.available();
        byte[] data = new byte[available];
        fis.read(data);
        fis.close();
        return data;
    }

    @Override
    protected Class<?> findClass(String name) throws ClassNotFoundException {
        try {
            byte[] bytes = loadByte(name);
            //defineClass converts a byte array into a class object. This byte array is the final byte array after the class file is read.
            return defineClass(name, bytes, 0, bytes.length);
        } catch (Exception e) {
            e.printStackTrace();
            throw new ClassNotFoundException();
        }
    }

    /**
     *  Rewrite the class loading method to implement its own loading logic and do not delegate it to parents
     * @param name
     * @param resolve
     * @return
     * @throws ClassNotFoundException
     */
    @Override
    protected Class<?> loadClass(String name, boolean resolve) throws ClassNotFoundException {
        synchronized (getClassLoadingLock(name)){
            // First, check if the class has already been loaded
            Class<?> loadedClass = findLoadedClass(name);
            if(loadedClass == null){
                // If still not found, then invoke findClass in order to find the class
                loadedClass = findClass(name);
                // this is the defining class loader; record the stats
                sun.misc.PerfCounter.getFindClassTime().addElapsedTimeFrom(System.nanoTime());
                sun.misc.PerfCounter.getFindClasses().increment();
            }
            if(resolve){
                resolveClass(loadedClass);
            }
            return loadedClass;
        }
    }

    public static void main(String[] args) throws Exception {
        NoParentClassLoader classLoader = new NoParentClassLoader("D:\\extra\\keepLearning\\TT\\SpringMVC\\JVMTT\\target\\classes");
        //Try to use your own rewriting class loading mechanism to load your own Java lang.String. class
        Class<?> clazz = classLoader.loadClass("java.lang.Integer");
//        Object obj = ;
        Method method = clazz.getMethod("sout");
        System.out.println(method.invoke(clazz.newInstance()));
        System.out.println(clazz.getClassLoader().getClass().getName());
    }
}



++++++++++++++++++++++++++++++++++++
Output results:
java.lang.SecurityException: Prohibited package name: java.lang
	at java.lang.ClassLoader.preDefineClass(ClassLoader.java:662)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:761)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:642)
	at com.jvm.tt.NoParentClassLoader.findClass(NoParentClassLoader.java:30)
	at com.jvm.tt.NoParentClassLoader.loadClass(NoParentClassLoader.java:51)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at com.jvm.tt.NoParentClassLoader.main(NoParentClassLoader.java:66)
Exception in thread "main" java.lang.ClassNotFoundException
	at com.jvm.tt.NoParentClassLoader.findClass(NoParentClassLoader.java:33)
	at com.jvm.tt.NoParentClassLoader.loadClass(NoParentClassLoader.java:51)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at com.jvm.tt.NoParentClassLoader.main(NoParentClassLoader.java:66)

When calling the custom findClass, the defineClass method will report the prohibited package name: Java Lang error, and then throw ClassNotFoundException to print the following ClassNotFoundException error.

Tomcat breaks the parental delegation mechanism:

reason:

1. As a web container, Tomcat needs to deploy two different sets of applications, and these two sets of applications will depend on different versions of the same third-party library, so the class library of each application is required to be relatively independent.

2. The same version of the same class library deployed in the same web container should be shared.

3. The web container has its own class library and is not confused with the application.

4. The web container should support the hot loading of jsp files, that is, after the application runs, the jsp files can be modified at any time, which can take effect immediately without restarting.

Tomcat custom class loader

Several main class loaders of tomcat:

commonClassLoader: the most basic class loader of tomcat. The classes in the loading path can be accessed by the tomcat container itself and various webapp s

Catalina classloader: the private classloader of tomcat container. The classes in the loading path are only visible to tomcat container, not webapp

Shared classloader: the class loader shared by each webapp. The classes in the loading path are visible to all webapps, but not to tomcat

WebappClassLoader: the private class loader of each webapp. The classes in the loading path are only visible to the current webapp. For example, load the classes in the war package. Each war package has its own WebappClassLoader to isolate each other.

As shown in the figure:

The classes that can be loaded by CommonClassLoader can be used by CatalinaClassLoader and SharedClassLoader. It is a common class library. The classes that CatalinaClassLoader and SharedClassLoader can load are isolated from each other.

WebappClassLoader can use the classes loaded by SharedClasLoader, but the class libraries loaded by each WebappClassLoader are isolated from each other.

The loading range of Jasper class loader is only the one compiled by this JSP file Class file. When the web container detects that the JSP file has been modified, it will replace the current JasperClassLoader instance, and then create a new JSP class loader to realize the hot loading function of JSP file.

Each WebappClassLoader loads the class file in its own directory and will not pass it to the parent class loader, breaking the parental delegation mechanism.

Simulate the WebappClassLoader of tomcat to load its own war package application, so as to realize the coexistence and isolation of different versions of class libraries in the application package

package com.jvm.tt;

import java.io.File;
import java.io.FileInputStream;
import java.lang.reflect.Method;

public class BreakParentClassLoader extends ClassLoader {
    private String classPath;

    public BreakParentClassLoader(String classPath) {
        this.classPath = classPath;
    }

    private byte[] loadByte(String name) throws Exception{
        name = name.replaceAll("\\.", "/");
        FileInputStream fis = new FileInputStream(new File(classPath + "/" + name + ".class"));
        int available = fis.available();
        byte[] data = new byte[available];
        fis.read(data);
        fis.close();
        return data;
    }

    @Override
    protected Class<?> findClass(String name) throws ClassNotFoundException {
        try {
            byte[] bytes = loadByte(name);
            //defineClass converts a byte array into a class object. This byte array is the final byte array after the class file is read.
            return defineClass(name, bytes, 0, bytes.length);
        } catch (Exception e) {
            e.printStackTrace();
            throw new ClassNotFoundException();
        }
    }

    /**
     *  Rewrite the class loading method to implement its own loading logic and do not delegate it to parents
     * @param name
     * @param resolve
     * @return
     * @throws ClassNotFoundException
     */
    @Override
    protected Class<?> loadClass(String name, boolean resolve) throws ClassNotFoundException {
        synchronized (getClassLoadingLock(name)){
            // First, check if the class has already been loaded
            Class<?> loadedClass = findLoadedClass(name);
            if(loadedClass == null){
                // If still not found, then invoke findClass in order to find the class
                long nanoTime = System.nanoTime();
                //if not self-define class, call parent class loader, else load class by self-define class loader
                if(!name.startsWith("com.jvm.tt")){
                    loadedClass = getParent().loadClass(name);
                }else{
                    loadedClass = findClass(name);
                }
                // this is the defining class loader; record the stats
                sun.misc.PerfCounter.getFindClassTime().addElapsedTimeFrom(System.nanoTime());
                sun.misc.PerfCounter.getFindClasses().increment();
            }
            if(resolve){
                resolveClass(loadedClass);
            }
            return loadedClass;
        }
    }

    public static void main(String[] args) throws Exception {
        BreakParentClassLoader classLoader = new BreakParentClassLoader("D:\\extra\\keepLearning\\TT\\SpringMVC\\JVMTT\\target\\classes");
        Class<?> clazz = classLoader.loadClass("com.jvm.tt.entity.Student");
        Object obj = clazz.newInstance();
        Method method = clazz.getMethod("studying");
        method.invoke(obj);
        System.out.println(clazz.getClassLoader());

        System.out.println("*******************************************");

        BreakParentClassLoader selfClassLoader = new BreakParentClassLoader("D:\\extra\\keepLearning\\TT\\SpringMVC\\JVMTT\\target\\classes");
        Class<?> selfClass = selfClassLoader.loadClass("com.jvm.tt.entity.Student");
        Object selfObj = selfClass.newInstance();
        Method selfMethod = selfClass.getMethod("studying");
        selfMethod.invoke(selfObj);
        System.out.println(selfClass.getClassLoader());

    }
}


+++++++++++++++++++++++++++++++++++++++++++++++
Operation results:

=======Load the class and call the method with your own loader=======
com.jvm.tt.BreakParentClassLoader@14ae5a5
*******************************************
=======Load the class and call the method with your own loader=======
com.jvm.tt.BreakParentClassLoader@135fbaa4


Note: in the same JVM, two class objects with the same path and the same class name can coexist. Their class loaders are different, so they are not the same class.

Topics: jvm