Explore Java non reflection mechanism and how to save startup memory resource consumption!

Posted by unibroue on Tue, 08 Mar 2022 02:00:46 +0100

As Java updates new features, such as architectural decisions and their requirements. At present, cloud computing usually requires applications to have better bootability in addition to less initial memory. Therefore, it is necessary to redesign the manufacturing method of the framework to eliminate the bottleneck of reflection.

Reflection plays an important role in the framework, whether it is the classic ORM or the REST API such as JAX-RS. By greatly reducing various operations, the work of Javaer becomes easier.

For end users (here refers to users using these frameworks), the whole process only needs to add some symbols in the class, and all operations can run normally. Their class metadata will be read and used to facilitate some processes. Currently, the most popular way to execute this type is through introspection, so that the dynamic language concept of Java can be easily produced.

Because a large number of resources have been created and examples and documentation of such work are provided, the use of reflection API s within the framework simplifies such work. However, for some reason, we are here to discuss two issues: startup application latency and memory consumption.

Start application delay: all processing and data structures will be executed at execution time. Imagine a dependency injection engine that requires step-by-step scanning, checking scope, dependencies, and so on. Therefore, the more categories need to be analyzed, the more processing is required, and the response time is greatly increased.

Memory consumption: each Class needs to traverse to search for metadata in the Class. A ReflectionData cache loads all the information of the Class, that is, to search for simple information such as getSimpleName(). All metadata information will be loaded and referenced through SoftReference, which takes some time to get out of memory.

In summary: the reflection method has problems with initial memory consumption and application startup latency. This is because data, analysis and parser processing are performed immediately after the application starts. As the number of classes increases, memory and runtime consumption tend to increase.

The solution to these problems is to make the framework perform these operations at compile time rather than at run time:

  • When the application starts, the metadata and system are ready.

  • There is no need to call reflection classes, including ReflectionData, which reduces memory consumption at startup.

  • There is no need to worry about the impact of erasure type.

Another point to avoid reflection is that we can use AoT more easily and create native code through GraalVM, which is an exciting possibility, especially for the serverless concept. The program runs once and then returns the entire resource to the operating system.


Demo code

After explaining the concept of reading type, the next step will be to create a simple tool that converts Java classes from some representation of the entity to be mapped to a Map. The property that will be converted and the field that will be a unique identifier. Let's do everything as shown in the following code:

@Documented@Target(ElementType.TYPE)@Retention(RetentionPolicy.RUNTIME)public @interface Entity {    String value() default "";}@Documented@Target(ElementType.FIELD)@Retention(RetentionPolicy.RUNTIME)public @interface Column {    String value() default "";}@Documented@Target(ElementType.FIELD)@Retention(RetentionPolicy.RUNTIME)public @interface Id {    String value() default "";}

In order to simplify the comparison with reflection or other options, an interface will be created, which is responsible for the conversion with the Map.

import java.util.Map;public interface Mapper {    <T>  T toEntity(Map<String, Object> map, Class<T> type);    <T> Map<String, Object> toMap(T entity);}

To compare the two solutions, the first implementation will be implemented through reflection. One point is that there are several strategies to deal with reflection. For example, use the "java.beans" package with Introspector in combination; However, in this example, we will operate in the simplest way to show how it works.

public class ReflectionMapper implements Mapper {    @Override    public <T> T toEntity(Map<String, Object> map, Class<T> type) {        Objects.requireNonNull(map, "Map is required");        Objects.requireNonNull(type, "type is required");        final Constructor<?>[] constructors = type.getConstructors();        try {            final T instance = (T) constructors[0].newInstance();            for (Field field : type.getDeclaredFields()) {                write(map, instance, field);            }            return instance;        } catch (InstantiationException | IllegalAccessException | InvocationTargetException exception) {            throw new RuntimeException("An error to field the entity process", exception);        }    }    @Override    public <T> Map<String, Object> toMap(T entity) {        Objects.requireNonNull(entity, "entity is required");        Map<String, Object> map = new HashMap<>();        final Class<?> type = entity.getClass();        final Entity annotation = Optional.ofNullable(                type.getAnnotation(Entity.class))                .orElseThrow(() -> new RuntimeException("The class must have Entity annotation"));        String name = annotation.value().isBlank() ? type.getSimpleName() : annotation.value();        map.put("entity", name);        for (Field field : type.getDeclaredFields()) {            try {                read(entity, map, field);            } catch (IllegalAccessException exception) {                throw new RuntimeException("An error to field the map process", exception);            }        }        return map;    }    private <T> void read(T entity, Map<String, Object> map, Field field) throws IllegalAccessException {        final Id id = field.getAnnotation(Id.class);        final Column column = field.getAnnotation(Column.class);        final String fieldName = field.getName();        if (id != null) {            String idName = id.value().isBlank() ? fieldName : id.value();            field.setAccessible(true);            final Object value = field.get(entity);            map.put(idName, value);        } else if (column != null) {            String columnName = column.value().isBlank() ? fieldName : column.value();            field.setAccessible(true);            final Object value = field.get(entity);            map.put(columnName, value);        }    }    private <T> void write(Map<String, Object> map, T instance, Field field) throws IllegalAccessException {        final Id id = field.getAnnotation(Id.class);        final Column column = field.getAnnotation(Column.class);        final String fieldName = field.getName();        if (id != null) {            String idName = id.value().isBlank() ? fieldName : id.value();            field.setAccessible(true);            final Object value = map.get(idName);            if (value != null) {                field.set(instance, value);            }        } else if (column != null) {            String columnName = column.value().isBlank() ? fieldName : column.value();            field.setAccessible(true);            final Object value = map.get(columnName);            if (value != null) {                field.set(instance, value);            }        }    }}

After building the mapper, the next step is to make a small example. So let's create an Animal entity.

@Entity("animal")public class Animal {    @Id    private String id;    @Column("native_name")    private String name;    public Animal() {    }    public Animal(String id, String name) {        this.id = id;        this.name = name;    }    public String getId() {        return id;    }    public void setId(String id) {        this.id = id;    }    public String getName() {        return name;    }    public void setName(String name) {        this.name = name;    }}public class ReflectionMapperTest {    private Mapper mapper;    @BeforeEach    public void setUp() {        this.mapper = new ReflectionMapper();    }    @Test    public void shouldCreateMap() {        Animal animal = new Animal("id", "lion");        final Map<String, Object> map = mapper.toMap(animal);        Assertions.assertEquals("animal", map.get("entity"));        Assertions.assertEquals("id", map.get("id"));        Assertions.assertEquals("lion", map.get("native_name"));    }    @Test    public void shouldCreateEntity() {        Map<String, Object> map = new HashMap<>();        map.put("id", "id");        map.put("native_name", "lion");        final Animal animal = mapper.toEntity(map, Animal.class);        Assertions.assertEquals("id", animal.getId());        Assertions.assertEquals("lion", animal.getName());    }}

In this way, the implementation of reflection implementation is demonstrated. If you want to use this type of tool in other projects, you can create a small project and add other dependencies, and these operations and reads will be performed at run time.

It should be noted that there are some options and policies that can be used in reflection. For example, create an internal cache of these metadata to avoid continuous use of ReflectionData or compiling classes at execution time from this information.

But most importantly, the whole process will take place at execution time. In order to move the process to compilation, we will use the Java Annotation Processor API.

To become an entity class in a process, you need to extend the AbstractProcessor class, use the SupportedAnnotationTypes annotation to define which classes will be read at compile time, and define the process method of the code core. This method will perform all analyses. The final step is to register the class as SPI and the code will be ready to run at compile time.

@SupportedAnnotationTypes("org.soujava.medatadata.api.Entity")public class EntityProcessor extends AbstractProcessor {//...     @Override    public boolean process(Set<? extends TypeElement> annotations,                           RoundEnvironment roundEnv) {        final List<String> entities = new ArrayList<>();        for (TypeElement annotation : annotations) {            roundEnv.getElementsAnnotatedWith(annotation)                    .stream().map(e -> new ClassAnalyzer(e, processingEnv))                    .map(ClassAnalyzer::get)                    .filter(IS_NOT_BLANK).forEach(entities::add);        }        try {            if (!entities.isEmpty()) {                createClassMapping(entities);                createProcessorMap();            }        } catch (IOException exception) {            error(exception);        }        return false;    }//... }

Importantly, the configuration of Java annotation processing requires more configuration steps than reflection. However, in the first step, the subsequent steps are often similar to the reflection API. You can use POM The annotationProcessorPaths tag in the XML file is used to complete the dependency on this library type. One big advantage is that these dependencies are only visible at compile scope. That is, you can add dependencies to generate classes, such as using Mustache, without having to worry about these dependencies at run time.

After the dependency is added to the project and executed, the class is generated in the target / generated sources folder. In this example, the generation of all classes is attributed to the Mustache project.

@Generated(value= "Soujava ClassMappings Generator", date = "2021-01-21T13:08:48.618494")public final class ProcessorClassMappings implements ClassMappings {    private final List<EntityMetadata> entities;    public ProcessorClassMappings() {        this.entities = new ArrayList<>();        this.entities.add(new org.soujava.metadata.example.PersonEntityMetaData());        this.entities.add(new org.soujava.metadata.example.AnimalEntityMetaData());        this.entities.add(new org.soujava.metadata.example.CarEntityMetaData());    }

Typically, the end-user functionality of this library will not change much because the user will continue to comment in the entity, but all processing logic has been brought to compile time@ Entity("animal")

public class Animal {    @Id    private String id;    @Column("native_name")    private String name;    public Animal() {    }    public Animal(String id, String name) {        this.id = id;        this.name = name;    }    public String getId() {        return id;    }    public void setId(String id) {        this.id = id;    }    public String getName() {        return name;    }    public void setName(String name) {        this.name = name;    }}public class ProcessorMapperTest {    private Mapper mapper;    @BeforeEach    public void setUp() {        this.mapper = new ProcessorMapper();    }    @Test    public void shouldCreateMap() {        Animal animal = new Animal("id", "lion");        final Map<String, Object> map = mapper.toMap(animal);        Assertions.assertEquals("animal", map.get("entity"));        Assertions.assertEquals("id", map.get("id"));        Assertions.assertEquals("lion", map.get("native_name"));    }    @Test    public void shouldCreateEntity() {        Map<String, Object> map = new HashMap<>();        map.put("id", "id");        map.put("native_name", "lion");        final Animal animal = mapper.toEntity(map, Animal.class);        Assertions.assertEquals("id", animal.getId());        Assertions.assertEquals("lion", animal.getName());    }}

Above, we discussed the advantages and disadvantages of reflection. This paper introduces an example with Java annotation processor, shows the advantages of Java AOT, and converts it to native.

Each option creates disadvantages. When you delete an application, all JIT optimizations are lost, and there is an old saying that with the development of technology, the JVM will be more efficient than native code. The definition of performance is very complex and does not only consider the application startup time.

Do you need a learning exchange group full of technical atmosphere? Just scan the code:

Topics: Java Programmer