springboot e-commerce project mall4j( https://gitee.com/gz-yami/mall4j)

java open source mall system

Environment: jdk1 eight

System: window/linux

fastjson version: 1.2.29

Key codes:

public class FastJsonUtil {
    /*
     * Convert pojo object to json string, and change hump naming to underline naming
     */
    public static String buildData(Object bean) {
        try {
            SerializeConfig config = new SerializeConfig();
            config.propertyNamingStrategy = PropertyNamingStrategy.SnakeCase;
            return JSON.toJSONString(bean, config);
        } catch (Exception e) {
            return null;
        }
    }
}

// An arbitrary entity class
public class T {
    public String userName;
    public String userArg;
    public String userGender;
    public String userAddr;

    public T(String userName, String userArg, String userGender, String userAddr) {
        this.userName = userName;
        this.userArg = userArg;
        this.userGender = userGender;
        this.userAddr = userAddr;
    }
}

Simulate concurrency (JMeter can also be used)

public class Main {
    public static void main(String[] args) throws InterruptedException {
        ThreadPoolExecutor poolExecutor = new ThreadPoolExecutor(6, 6, 1, TimeUnit.SECONDS, new ArrayBlockingQueue<>(999999));
        for (;;) {
            // Submit the task every 1 millisecond
            TimeUnit.MILLISECONDS.sleep(1);
            poolExecutor.submit(Main::prcess);
        }
    }

    // Simulate a request call, and finally convert a t object into json format to return
    public static void prcess() {
        T t = new T("1", "2", "3", "4");
        String res = FastJsonUtil.buildData(t);
        // System.out.printf(res);
    }
}

jvm startup parameters:

-Xms4g -Xmx4g -Xmn2g -Xss1024K -XX:+UseConcMarkSweepGC (limit heap memory size, adjust the ratio of new era area to old generation to 1:1, set thread stack size, and use cms garbage collector)

After running for a period of time, you will find that the memory occupied by the java program is far greater than 4g.

Since the heap memory size has been set to be within 4g, but the memory occupied by the program is far more than 4g, which directly climbs to the physical memory, it may be caused by the leakage of off heap memory (direct memory or metaspace memory).

In the program code, there is no direct memory operation, and no io related framework such as netty is used.

In order to more accurately locate whether it is direct memory or metaspace, add a startup parameter - XX:MaxDirectMemorySize=1g. Restart the project. It is found that there is no effect, and the problem still exists.

So lock the problem to the matespace and use jmap -heap pid to check the heap usage:

It can be found that MaxMetaspaceSize is the memory size of the system without any restriction. So it may be because after calling a fastjson method, it handles some things and needs to store some things in metaspace. Because metaspace does not limit the memory size, java programs occupy more than 4g of memory and keep rising, which will eventually trigger a memory warning.

Try adding - XX:MetaspaceSize=256m -XX:MaxMetaspaceSize=512m to the jvm startup parameter (generally, these two values can be set to the same and will not be repeated). Start the project again and simulate concurrency. It is found that the memory occupation is normal and does not exceed the limit.

Therefore, at this step, it can be determined that fastjson may always load a class during processing, resulting in excessive memory consumption of metaspace.

After adding - verbose:class to the startup parameter, start the project again and observe which class is loaded repeatedly

[Loaded com.alibaba.fastjson.serializer.ASMSerializer_1_T from file:/E:/maven/repository/com/alibaba/fastjson/1.2.29/fastjson-1.2.29.jar]
[Loaded com.alibaba.fastjson.serializer.ASMSerializer_1_T from file:/E:/maven/repository/com/alibaba/fastjson/1.2.29/fastjson-1.2.29.jar]
[Loaded com.alibaba.fastjson.serializer.ASMSerializer_1_T from file:/E:/maven/repository/com/alibaba/fastjson/1.2.29/fastjson-1.2.29.jar]
[Loaded com.alibaba.fastjson.serializer.ASMSerializer_1_T from file:/E:/maven/repository/com/alibaba/fastjson/1.2.29/fastjson-1.2.29.jar]
...

To sum up, it is obvious that asmserializer has been loading_ 1_ T this class.

The next step is to find out where fastjson has been repeatedly loading this class.

From JSON Start with the tojsonstring method and enter com alibaba. fastjson. JSON #tojsonstring (java.lang.object, com.alibaba.fastjson.serializer.serializeconfig, com.alibaba.fastjson.serializer.serializefilter [], java.lang.string, int, com.alibaba.fastjson.serializer.serializerfeature...) method

Then go to getobjectwriter (clazz) - > config Getobjectwriter (clazz) - > put (clazz, createjavabeanserializer (clazz)) - > createjavabeanserializer (beaninfo) - > createasmserializer (beaninfo) method

In createasm serializer, there are several lines of code

......
// Splice class name
String className = "ASMSerializer_" + seed.incrementAndGet() + "_" + clazz.getSimpleName();
String packageName = ASMSerializerFactory.class.getPackage().getName();
String classNameType = packageName.replace('.', '/') + "/" + className;
String classNameFull = packageName + "." + className;
ClassWriter cw = new ClassWriter();
// Then ASMSerializer is loaded here_ Class of
cw.visit(V1_5 //
         , ACC_PUBLIC + ACC_SUPER //
         , classNameType //
         , JavaBeanSerializer //
         , new String[] { ObjectSerializer } //
);
......

Therefore, here it is. Every time it is called here, it will load ASMSerializer_1_T into metaspace.

This part of the code is in ASMSerializerFactory.

Back to the user code, in the line of SerializeConfig config = new SerializeConfig(), enter the parameterless structure of SerializeConfig, and its parametered structure will be called. There are a few lines of code in the parameterized construct

if (asm) {
    asmFactory = new ASMSerializerFactory();
}

Everything is clear. Because SerializeConfig is created all the time, ASMSerializerFactory will also be created repeatedly. Then, when ASMSerializerFactory calls the createASMSerializer method in this class, com. Will be loaded repeatedly alibaba. fastjson. serializer. ASMSerializer_ 1_ T

terms of settlement:

Set SerializeConfig config = new SerializeConfig();config.propertyNamingStrategy = PropertyNamingStrategy.SnakeCase; These two lines of code are mentioned outside the method, and the transformation is as follows:

public class FastJsonUtil {
     private final static SerializeConfig config = new SerializeConfig();
     static {
         config.propertyNamingStrategy = PropertyNamingStrategy.SnakeCase;
     }
    /*
     * Convert pojo object to json string, and change hump naming to underline naming
     */
    public static String buildData(Object bean) {
        try {
            return JSON.toJSONString(bean, config);
        } catch (Exception e) {
            return null;
        }
    }
}

Restart the project, the problem is solved, and asmserializer will not be loaded all the time_ 1_ T this class.

Use JVisualVm to observe memory activity

metaspace before modification:

Modified metaspace:

Obviously, it can be observed that after the modification, the class and metaspace memory will not be loaded frequently, and the situation will not rise sharply.

springboot e-commerce project mall4j( https://gitee.com/gz-yami/mall4j)

java open source mall system