Using YANG-UTIL to bring classes into ODL can't load BUG

Posted by nominator on Sun, 12 May 2019 07:06:22 +0200

ODL is based on MDSL, whose model is defined in YAGN language, and the class generated by Yang-tool is used in programming. The data stored in ODL's DataStore needs to be serialized and deserialized. The serialized deserialized code is generated dynamically through Javassist.

BindingToNormalizedNodecodec.java

    public <T extends DataObject> Entry<YangInstanceIdentifier, NormalizedNode<?, ?>> toNormalizedNode(InstanceIdentifier<T> path, T data) {
        return this.codecRegistry.toNormalizedNode(path, data);
    }

As can be seen from the figure above, the model code generated by YANG tool is serialized into a general Normalized Node structure with an example constructed by the code. The tool exists in Cache

private final LoadingCache<Class<? extends DataObject>, DataObjectSerializer> serializers;
this.serializers = CacheBuilder.newBuilder().weakKeys().build(new GeneratorLoader());

There are various definitions of YANG files in the project. It is impossible to submit each structure to define Serializer, so the corresponding Serializer needs to be generated dynamically according to the type.

AbstractStreamWriterGenerator#load

            Class<? extends DataObjectSerializerImplementation> cls;
            try {
                cls = (Class<? extends DataObjectSerializerImplementation>) ClassLoaderUtils
                        .loadClass(type.getClassLoader(), serializerName);
            } catch (final ClassNotFoundException e) {
                cls = generateSerializer(type, serializerName);
            }

Refer to generateSerializer in the generated code

The type is YangProtectRestoreParameter, and the final class returned is YangProtectRestoreParameter $StreamWriter (dynamically generated)

The generated entry is AbstractStream Writer Generator #generateEmitter0

private CtClass generateEmitter0(final Class<?> type, final DataObjectSerializerSource source, final String serializerName) {
        final CtClass product;

        /*
         * getSerializerBody() has side effects, such as loading classes and codecs, it should be run in model class
         * loader in order to correctly reference load child classes.
         *
         * Furthermore the fact that getSerializedBody() can trigger other code generation to happen, we need to take
         * care of this before calling instantiatePrototype(), as that will call our customizer with the lock held,
         * hence any code generation will end up being blocked on the javassist lock.
         */
        final String body = ClassLoaderUtils.withClassLoader(type.getClassLoader(), new Supplier<String>() {
                @Override
                public String get() {
                    return source.getSerializerBody().toString();
                }
            }
        );

        try {
            product = javassist.instantiatePrototype(DataObjectSerializerPrototype.class.getName(), serializerName, new ClassCustomizer() {
                @Override
                public void customizeClass(final CtClass cls) throws CannotCompileException, NotFoundException {
                    // Generate any static fields
                    for (final StaticConstantDefinition def : source.getStaticConstants()) {
                        final CtField field = new CtField(javassist.asCtClass(def.getType()), def.getName(), cls);
                        field.setModifiers(Modifier.PRIVATE + Modifier.STATIC);
                        cls.addField(field);
                    }

                    // Replace serialize() -- may reference static fields
                    final CtMethod serializeTo = cls.getDeclaredMethod(SERIALIZE_METHOD_NAME, serializeArguments);
                    serializeTo.setBody(body);

                    // The prototype is not visible, so we need to take care of that
                    cls.setModifiers(Modifier.setPublic(cls.getModifiers()));
                }
            });
        } catch (final NotFoundException e) {
            LOG.error("Failed to instatiate serializer {}", source, e);
            throw new LinkageError("Unexpected instantation problem: serializer prototype not found", e);
        }
        return product;
    }

The above code uses Javassist

Javassist is a dynamic class library that can be used to check, "dynamically" modify and create Java classes. Its function is similar to that of jdk, but stronger than that of jdk.

Here's how to reproduce the Bug through a test case

First of all, let's look at the details of Forwarding we defined. Note that it's still in a package with the lowercase forward command. The directory structure is as follows:

The test case code is as follows:

package com.zte.sunquan.demo.cache;

import javassist.ClassPool;
import javassist.CtClass;
import javassist.CtConstructor;
import javassist.CtField;
import org.junit.Test;

/**
 * Created by sunquan on 2018/8/21.
 */
public class JavaAssistTestBug {

    @Test
    public void testBug() throws Exception {
        //Create a class dynamically
        ClassPool pool = ClassPool.getDefault();
        CtClass cc = pool.makeClass("com.zte.sunquan.demo.User");
        //Create properties
        CtField field03 = CtField.make("private org.opendaylight.yang.gen.v1.cx.zx.xxx.model.cim.configuration.rev170216.forwarding.A a;", cc);
        CtField field04 = CtField.make("private org.opendaylight.yang.gen.v1.cx.zx.xxx.model.cim.configuration.rev170216.Forwarding b;", cc);
        cc.addField(field03);
        cc.addField(field04);

        //Creating method
//        CtMethod method01 = CtMethod.make("public org.opendaylight.yang.gen.v1.cx.zx.xx.model.cim.configuration.rev170216.forwarding.A getName(){return name;}", cc);
//        CtMethod method02 = CtMethod.make("public void setName(String name){this.name = name;}", cc);
//        cc.addMethod(method01);
//        cc.addMethod(method02);
        //Add a parametric constructor
        CtConstructor constructor = new CtConstructor(new CtClass[]{CtClass.intType, pool.get("java.lang.String")}, cc);
        constructor.setBody("{this.b=new org.opendaylight.yang.gen.v1.cx.zx.xxx.model.cim.configuration.rev170216.forwarding();}");
        cc.addConstructor(constructor);
        //Parametric constructor
//        CtConstructor cons = new CtConstructor(null, cc);
//        cons.setBody("{}");
//        cc.addConstructor(cons);

        cc.writeFile("E:/workspace/TestCompiler/src");
    }
}

The code logic is very simple. It creates a class User dynamically through Javassist, and defines two attributes and a constructor. It initializes b in the constructor, but it is important to note that it uses lowercase forwarding, but there is no lowercase forwarding class in fact.

Code execution throws an exception:

java.lang.RuntimeException: cannot find org.opendaylight.yang.gen.v1.cx.zx.xxx.model.cim.configuration.rev170216.forwarding: org.opendaylight.yang.gen.v1.cx.zx.xxx.model.cim.configuration.rev170216.Forwarding found in org/opendaylight/yang/gen/v1/cx/zx/xxx/model/cim/configuration/rev170216/forwarding.class
 

Exception information indicates that forwarding.class cannot be found

A close look at the call stack reveals that:

Real exception throws are caused by inconsistencies between cf.getName() and qualifiedName

Why qualified Name is lowercase forwarding? Judging from the code logic, if it is executed to this point, forwarding.class can actually be found in classPool. Verification:

The code goes to return new CtClassType, so the find(classname) return is not empty, but the classname is actually:

org.opendaylight.yang.gen.v1.cx.zx.xxx.model.cim.configuration.rev170216.forwarding

So you actually found the forwarding.class class class from AppClassLoader? This is clearly a bag, can not be a class, too weird!

Looking at the directory again, it's clear in the classPath that there's only Forwarding.class

 

It's time to expose the fact that AppClassLoader doesn't distinguish between case and case when it finds a class in Windows, so what it finds is actually Forwarding.class. If it uses the acquired handle to perform the operation at this time, there's really no problem.

But Javassist will further determine whether the class name is consistent, you pass in forwarding, and according to JAVA development habits, the class name can only be capitalized on and off, so Javassist will make a one-step conversion to capitalize the first letter (in the ClassFile constructor).

thisclassname = constPool.getClassInfo(thisClass);

So there is inconsistency, resulting in exception throwing.

 

Amendment:

1. Do not name classes with package names

2. Assisti source code modification to determine whether it is a class or a path

Topics: Java JDK Programming Junit