In depth Dubbo - comparison, application and source code analysis of Dubbo SPI & Java SPI

Posted by GRUMBLE on Wed, 19 Jan 2022 01:50:17 +0100

Tip: after the article is written, the directory can be generated automatically. Please refer to the help document on the right for how to generate it

catalogue

preface

1, Brain map of Dubbo SPI & Java SPI

2, Java SPI

1. Code application

2. Source code analysis

3. Problem analysis

summary

preface

What is SPI? What problems have been solved? Why does Dubbo implement SPI? What's wrong with Java SPI?

After reading this article, you should have a general understanding of this. As the core mechanism of Dubbo, understanding the use and implementation principle of Dubbo SPI will be very helpful for us to deeply understand the overall design idea of Dubbo.

Tip: the following is the main content of this article. The following cases can be used for reference

1, Brain map of Dubbo SPI & Java SPI

Let's start with the figure above to facilitate you to quickly locate the required content.

2, Java SPI

1. Code application

Define interface classes:

public interface JavaSPIInterface {
    String echo(String msg);
}

Define implementation classes:

// The following three implementation classes should be divided into three Java files, which I put in a code segment for convenience
// Implementation class 1
public class JavaSPIInterfaceImpl1 implements JavaSPIInterface {
    @Override
    public String echo(String msg) {
        return "JavaSPIInterfaceImpl1 -- " + msg;
    }
}

// Implementation class 2
public class JavaSPIInterfaceImpl2  implements JavaSPIInterface {
    @Override
    public String echo(String msg) {
        return "JavaSPIInterfaceImpl2 -- " + msg;
    }
}

// Implementation class 3
public class JavaSPIInterfaceImpl3 implements JavaSPIInterface {
    @Override
    public String echo(String msg) {
        return "JavaSPIInterfaceImpl3 -- " + msg;
    }
}

Configure the extension point. The configuration file is saved in meta-inf / services / com xxx. Javaspi interface (i.e. the full path of the interface):

# Full path of extension point interface implementation
com.xxx.impl.JavaSPIInterfaceImpl1
com.xxx.impl.JavaSPIInterfaceImpl2
com.xxx.impl.JavaSPIInterfaceImpl3

Unit test class

ServiceLoader<JavaSPIInterface> javaSPIInterfaces = ServiceLoader.load(JavaSPIInterface.class);        
Iterator<JavaSPIInterface> iterator = javaSPIInterfaces.iterator();
while (iterator.hasNext()){
   System.out.println(iterator.next().echo("sks-9527"));
}

The above four steps are the core process of Java SPI.

2. Source code analysis

ServiceLoader.load(JavaSPIInterface.class); Source code analysis:

public static <S> ServiceLoader<S> load(Class<S> service,
                                            ClassLoader loader)
    {
        return new ServiceLoader<>(service, loader);
    }
.....
private ServiceLoader(Class<S> svc, ClassLoader cl) {
        service = Objects.requireNonNull(svc, "Service interface cannot be null");
        loader = (cl == null) ? ClassLoader.getSystemClassLoader() : cl;
        acc = (System.getSecurityManager() != null) ? AccessController.getContext() : null;
        // Core method, reload extension point configuration
        reload();
    }
.....
public void reload() {
        providers.clear();
        //Finally, a lazyitterer instance is created. At this time, the extension point file is loaded and the extension point implementation class is instantiated
        lookupIterator = new LazyIterator(service, loader);
    }
.....
private LazyIterator(Class<S> service, ClassLoader loader) {
            this.service = service;
            this.loader = loader;
        }

Lazyitrator #hasnext source code analysis:

public Iterator<S> iterator() {
        return new Iterator<S>() {
            // Get the iterator of SerivceLoader property LinkedHashMap < string, s > providers,
            //LinkedHashMap < string, s > providers are used to store instances of extension points (key is the full path of the implementation class, and value is the corresponding instance)
            Iterator<Map.Entry<String,S>> knownProviders
                = providers.entrySet().iterator();

            public boolean hasNext() {
                // knownProviders.hasNext() is true, which indicates that there is data in the cache and there is no need for lookupIterator to load again.
                // If it is false, you need to load the extension point configuration to determine whether the next instance exists.
                if (knownProviders.hasNext())
                    return true;
                // Load extension point profile
                return lookupIterator.hasNext();
            }

            public S next() {
                // If there is data in the cached instance, it will be returned directly to avoid repeated instantiation of extension point implementation
                if (knownProviders.hasNext())
                    return knownProviders.next().getValue();
                // Load extension point implementation
                return lookupIterator.next();
            }

            public void remove() {
                throw new UnsupportedOperationException();
            }

        };
    }

.......
// lookupIterator.hasNext();  Finally, the following code is called
private boolean hasNextService() {
     if (nextName != null) {
        return true;
     }
     // Enumeration < URL > file resources of the configure extension point,
     // If it is null, load it. If it is not null, get the content of the extension point file
     // Obviously -- [thread unsafe operation]
     if (configs == null) {
         try {
           String fullName = PREFIX + service.getName();
           if (loader == null)
              configs = ClassLoader.getSystemResources(fullName);
           else
               configs = loader.getResources(fullName);
          } catch (IOException x) {
                    fail(service, "Error locating configuration files", x);
           }
      }
      // The pending extension point configures the iterator for the full path string of the class
      while ((pending == null) || !pending.hasNext()) {
         if (!configs.hasMoreElements()) {
             return false;
         }
         // Parsing profile content
         pending = parse(service, configs.nextElement());
      }
         // Get the next full path string of the extension point implementation class,
         // In lookupIterator#next, the extension point will be implemented according to nextName to instantiate the class
         nextName = pending.next();
         return true;
}
.....
// Load extension point profile content
private Iterator<String> parse(Class<?> service, URL u)
        throws ServiceConfigurationError
    {
        InputStream in = null;
        BufferedReader r = null;
        // Stores the string of each line of the extension point configuration file (the full path of the extension point implementation class)
        ArrayList<String> names = new ArrayList<>();
        try {
            in = u.openStream();
            r = new BufferedReader(new InputStreamReader(in, "utf-8"));
            int lc = 1;
            while ((lc = parseLine(service, u, r, lc, names)) >= 0);
        } catch (IOException x) {
            fail(service, "Error reading configuration file", x);
        } finally {
            try {
                if (r != null) r.close();
                if (in != null) in.close();
            } catch (IOException y) {
                fail(service, "Error closing configuration file", y);
            }
        }
        // Returns the iterator of the full path collection of extension points
        return names.iterator();
    }

LazyIterator#nex

 public Iterator<S> iterator() {
        return new Iterator<S>() {

            Iterator<Map.Entry<String,S>> knownProviders
                = providers.entrySet().iterator();

            public boolean hasNext() {
                if (knownProviders.hasNext())
                    return true;
                return lookupIterator.hasNext();
            }

            public S next() {
                // If the cached instance already exists, the extension point instantiation will not be repeated
                if (knownProviders.hasNext())
                    return knownProviders.next().getValue();
                return lookupIterator.next();
            }

            public void remove() {
                throw new UnsupportedOperationException();
            }

        };
    }
....
// lookupIterator. After next () analysis, nextService will be called to instantiate the class
private S nextService() {
            if (!hasNextService())
                throw new NoSuchElementException();
            // nextName is the full path string of the next extension point in lazyitteror#hasnext
            String cn = nextName;
            nextName = null;
            Class<?> c = null;
            try {
                // String loaded as Class
                c = Class.forName(cn, false, loader);
            } catch (ClassNotFoundException x) {
                fail(service,
                     "Provider " + cn + " not found");
            }
            if (!service.isAssignableFrom(c)) {
                fail(service,
                     "Provider " + cn  + " not a subtype");
            }
            // Obviously -- [thread unsafe operation]
            try {
                // Call the default constructor to instantiate and convert it to an extension point interface
                S p = service.cast(c.newInstance());
                // Example of cache extension point interface implementation
                providers.put(cn, p);
                return p;
            } catch (Throwable x) {
                fail(service,
                     "Provider " + cn + " could not be instantiated",
                     x);
            }
            throw new Error();          // This cannot happen
        }

The core process of Java SPI is to load the extension point configuration file and judge whether there is a next extension point implementation class. The following figure is the flow chart of the hasNext method of the iterator returned by ServiceLoader.
The code structure is shown in the figure below

3. Problem analysis

resource consumption
1. When Java SPI loads the extension point implementation class, it cannot obtain the extension point implementation according to the actual needs. Only after loading and instantiating all the extension point implementations can it obtain the implementation class it needs. If there are instances in the implementation class that consume special resources but are not used in practical applications, it will be a waste of resources.
Lack of flexibility
1. In fact, the reason is the same as above. Only after loading all the implementations of the extension point can we judge the actual required implementation classes.

Thread unsafe

In the source code analysis of Java SPI, thread unsafe operations in comments have response text prompts.

Modify the extension point test class as follows

ServiceLoader<JavaSPIInterface> javaSPIInterfaces = ServiceLoader.load(JavaSPIInterface.class);
new Thread(() -> {
    Iterator<JavaSPIInterface> iterator = javaSPIInterfaces.iterator();
    while (iterator.hasNext()){
          System.out.println(iterator.next().echo("sks-9527"));
    }
}).start();
Iterator<JavaSPIInterface> iterator = javaSPIInterfaces.iterator();
while (iterator.hasNext()){
    System.out.println(iterator.next().echo("sks-9527"));
}

The results of multiple runs are as follows/
java.util.NoSuchElementException
	at sun.misc.CompoundEnumeration.nextElement(CompoundEnumeration.java:59)
	at java.util.ServiceLoader$LazyIterator.hasNextService(ServiceLoader.java:357)
	at java.util.ServiceLoader$LazyIterator.hasNext(ServiceLoader.java:393)
	at java.util.ServiceLoader$1.hasNext(ServiceLoader.java:474)
	at com.xxx.JavaSPITest.testJavaSpi(JavaSPITest.java:24)
// reason
 because java.util.ServiceLoader#The iterator returned by the iterator shares a lazyitterer lookupiterator instance,
And in LazyIterator Properties in Enumeration<URL> configs It is also shared and non thread safe.

3, Dubbo SPI

1. Code application

Custom interface

// @SPI tag this interface is a Dubbo SPI interface, and the value value represents the key of the default implementation of this SPI
// The value value of the annotation corresponds to the key of the extension point configuration file
@SPI("impl1")
public interface DubboSPIInterface {
    String echo(String message);
}

Custom implementation class

// The following three classes are three Java classes
public class DubboSPIInterfaceImpl1 implements DubboSPIInterface {
    @Override
    public String echo(String message) {
        return "DubboSPIInterfaceImpl1 -- " + message;
    }
}

public class DubboSPIInterfaceImpl2 implements DubboSPIInterface {
    @Override
    public String echo(String message) {
        return "DubboSPIInterfaceImpl2 -- " + message;
    }
}

public class DubboSPIInterfaceImpl3 implements DubboSPIInterface {
    @Override
    public String echo(String message) {
        return "DubboSPIInterfaceImpl3 -- " + message;
    }
}

Configuration file, extension point loading directory of Dubbo SPI. From the brain diagram, we can see that there are three directories: META-INF/services /, META-INF/dubbo /, META-INF/dubbo/internal /, and the name of the configuration file is the full class name of the interface.
```
impl1=com.iths.common.dubbo.spi.impl.DubboSPIInterfaceImpl1
impl2=com.iths.common.dubbo.spi.impl.DubboSPIInterfaceImpl2
impl3=com.iths.common.dubbo.spi.impl.DubboSPIInterfaceImpl3
```

Test code

@Test
    public void testDefaultExtension(){
        // Gets the default extension point implementation
        DubboSPIInterface defaultExtension = ExtensionLoader.getExtensionLoader(DubboSPIInterface.class).getDefaultExtension();
        System.out.println( defaultExtension.echo("sks-9527"));;
    }

    @Test
    public void testGetExtensionByKey(){
        // Or an extension point implementation with a specified name
        DubboSPIInterface defaultExtension = ExtensionLoader.getExtensionLoader(DubboSPIInterface.class).getExtension("impl2");
        System.out.println( defaultExtension.echo("sks-9527"));;
    }

summary

Tip: here is a summary of the article:
For example, the above is what we want to talk about today. This paper only briefly introduces the use of pandas, which provides a large number of functions and methods that enable us to process data quickly and conveniently.

Topics: Dubbo

Programmer Think

In depth Dubbo - comparison, application and source code analysis of Dubbo SPI & Java SPI

preface

1, Brain map of Dubbo SPI & Java SPI

2, Java SPI

1. Code application

2. Source code analysis

3. Problem analysis

3, Dubbo SPI

1. Code application

summary

Hot Topics