01 RPC overview

The following picture, which many small partners have seen, is a picture on Dubbo's official website describing the evolution of the project architecture.

02 environment preset

Step 1: first, we set up the project environment and create the pom.xml configuration file as follows:

 <dependency>
      <groupId>io.netty</groupId>
      <artifactId>netty-all</artifactId>
      <version>4.1.6.Final</version>
    </dependency>
    <dependency>
      <groupId>org.projectlombok</groupId>
      <artifactId>lombok</artifactId>
      <version>1.16.10</version>
    </dependency>

Step 2: create the project structure.

Before there was no RPC framework, our service calls were as follows:

It can be seen from the above figure that the call of the interface has no rules to follow. If you want to adjust it, you can adjust it. This leads to a certain stage of business development, the maintenance of the interface becomes very difficult. So some people put forward the concept of service governance. All services are not allowed to be directly called, but first register in the registration center, and then the registration center will coordinate and manage the status of all services and publish them to the public. The caller only needs to remember the service name and go to the registration center to obtain services. In this way, the management of services is greatly standardized, and the controllability of all servers can be improved. In fact, the whole design idea can also find living cases in our life. For example: we usually use IM tools for work communication, rather than face-to-face shouting. You just need to remember each other's numbers (such as Tencent QQ) provided by operators (that is, registration center). Another example: when we call, all phone numbers are assigned by the operator. When we need to talk to someone, we only need to dial the number of the other party, and the operator (Registration Center, such as China Mobile, China Unicom, China Telecom) will help us transfer the signal.

At present, the popular RPC service governance frameworks mainly include Dubbo and Spring Cloud. Let me take the classic Dubbo as an example. There are four core modules of Dubbo: Registry registration center, Provider server, Consumer and Monitor monitoring center, as shown below:

Registry
Consumer
Server (Provider)
Monitoring center

For convenience, we put all modules into one project. The main modules include:

Next, let's build the project structure. The specific screenshot of the project structure is as follows:

03 code practice

3.1 create API module

First, create the API module. Both provider and consumer follow the specification of the API module. To simplify, create two Service interfaces, which are:

package com.xinfan.netty.rpc.api;

/**
 * IRpcHelloService
 *
 * @author Lss
 * @date 2020/2/18 22:50
 * @Version 1.0
 */
public interface IRpcHelloService {
    String hello(String name);
}

Create the IRpcService interface, and complete the addition, subtraction, multiplication, and division operations of analog services. The specific code is as follows:

package com.xinfan.netty.rpc.api;

/**
 * IRpcService
 *
 * @author Lss
 * @date 2020/2/18 22:51
 * @Version 1.0
 */
public interface IRpcService {
    /** plus */
    public int add(int a,int b);
    /** reduce */
    public int sub(int a,int b);
    /** ride */
    public int mult(int a,int b);
    /** except */
    public int div(int a,int b);
}

At this point, the API module definition is complete, very simple. Next, we need to determine the transmission rules, that is, the transmission protocol. Of course, the content of the protocol must be customized to reflect the advantages of Netty.

3.2 create a custom agreement

The built-in HTTP protocol in Netty requires HTTP codec and decoder to complete the parsing. Let's see how to set the custom protocol?

To complete a custom protocol in Netty, it's very simple. You only need to define a common Java class. At present, handwritten RPC is mainly used for remote calling of Java code (similar to RMI, you should be familiar with it). What content of remote calling java code must be transmitted by the network? For example, service name? Which method of the service needs to be called? What are the arguments to the method? All these information needs to be transmitted to the server through the client.

Let's look at the specific code implementation and define the InvokerProtocol class:

import lombok.Data;

import java.io.Serializable;
/**
 * InvokerProtocol
 *
 * @author Lss
 * @date 2020/2/18 22:54
 * @Version 1.0
 */
@Data
public class InvokerProtocol implements Serializable {

    //Class name
    private String className;
    //Function name (method name)
    private String methodName;
    //Parameter type
    private Class<?>[] parames;
    //parameter list
    private Object[] values;
}

From the above code, we can see that the main information contained in the protocol includes class name, function name, parameter list and argument list, through which we can locate a specific business logic implementation.

3.3 implement the service logic of Provider server

We implement all the functions defined in the API in the provider module, and create two implementation classes respectively:

Rphelloserviceimpl class:

package com.xinfan.netty.rpc.provider;

import com.xinfan.netty.rpc.api.IRpcHelloService;

/**
 * RpcHelloServiceImpl
 *
 * @author Lss
 * @date 2020/2/18 23:00
 * @Version 1.0
 */
public class RpcHelloServiceImpl implements IRpcHelloService{

    @Override
    public String hello(String name) {
        return "Hello " + name + "!";
    }
}

RpcServiceImpl class:

package com.xinfan.netty.rpc.provider;

import com.xinfan.netty.rpc.api.IRpcService;

/**
 * RpcServiceImpl
 *
 * @author Lss
 * @date 2020/2/18 23:00
 * @Version 1.0
 */
public class RpcServiceImpl implements IRpcService {
    @Override
    public int add(int a, int b) {
        return a + b;
    }

    @Override
    public int sub(int a, int b) {
        return a - b;
    }

    @Override
    public int mult(int a, int b) {
        return a * b;
    }

    @Override
    public int div(int a, int b) {
        return a / b;
    }
}

3.4 complete registration of Registry service

The main function of registry is to register the service names and service reference addresses of all providers into a container and publish them to the public. Registry should start an external service. Obviously, it should serve as the server and provide an external accessible port. First, start a Netty service and create the RpcRegistry class. The specific code is as follows:

package com.xinfan.netty.rpc.registry;

import io.netty.bootstrap.ServerBootstrap;

import io.netty.channel.*;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioServerSocketChannel;
import io.netty.handler.codec.LengthFieldBasedFrameDecoder;
import io.netty.handler.codec.LengthFieldPrepender;
import io.netty.handler.codec.serialization.ClassResolvers;
import io.netty.handler.codec.serialization.ObjectDecoder;
import io.netty.handler.codec.serialization.ObjectEncoder;




/**
 * RpcRegistry
 *
 * @author Lss
 * @date 2020/2/18 23:27
 * @Version 1.0
 */
public class RpcRegistry {

    private int port;
    public RpcRegistry(int port){
        this.port = port;
    }
    public void start(){
        EventLoopGroup bossGroup = new NioEventLoopGroup();
        EventLoopGroup workerGroup = new NioEventLoopGroup();

        try {
            ServerBootstrap b = new ServerBootstrap();
            b.group(bossGroup, workerGroup)
                    .channel(NioServerSocketChannel.class)
                    .childHandler(new ChannelInitializer<SocketChannel>() {

                        @Override
                        protected void initChannel(SocketChannel ch) throws Exception {
                            ChannelPipeline pipeline = ch.pipeline();
                            //Custom protocol decoder
                            /** There are 5 input parameters, which are explained as follows
                             maxFrameLength: The maximum length of the frame. If the length of the frame is greater than this value, a TooLongFrameException is thrown.
                             lengthFieldOffset: Offset of length field: the position of the corresponding length field in the whole message data
                             lengthFieldLength: The length of the length field. For example, if the length field is int type, then the value is 4 (long type is 8)
                             lengthAdjustment: Compensation value to add to length field value
                             initialBytesToStrip: Number of first bytes removed from decoded frame
                             */
                            pipeline.addLast(new LengthFieldBasedFrameDecoder(Integer.MAX_VALUE, 0, 4, 0, 4));
                            //Custom protocol encoder
                            pipeline.addLast(new LengthFieldPrepender(4));
                            //Object parameter type encoder
                            pipeline.addLast("encoder",new ObjectEncoder());
                            //Object parameter type decoder
                            pipeline.addLast("decoder",new ObjectDecoder(Integer.MAX_VALUE,ClassResolvers.cacheDisabled(null)));                            
                            pipeline.addLast(new RegistryHandler());
                        }
                    })
                    .option(ChannelOption.SO_BACKLOG, 128)
                    .childOption(ChannelOption.SO_KEEPALIVE, true);
            ChannelFuture future = b.bind(port).sync();
            System.out.println("GP RPC Registry start listen at " + port );
            future.channel().closeFuture().sync();
        } catch (Exception e) {
            bossGroup.shutdownGracefully();
            workerGroup.shutdownGracefully();
        }
    }


    public static void main(String[] args) throws Exception {
        new RpcRegistry(8080).start();
    }
}

The specific logic of registration is implemented in the RegistryHandler. The above code mainly realizes the function of service registration and service invocation. Because all modules are created in the same project, in order to simplify, the server does not use remote call, but directly scans the local Class, and then uses reflection call. The code implementation is as follows:

package com.xinfan.netty.rpc.registry;

import com.xinfan.netty.rpc.protocol.InvokerProtocol;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelInboundHandlerAdapter;


import java.io.File;
import java.io.FileInputStream;
import java.io.UnsupportedEncodingException;
import java.lang.reflect.Method;
import java.net.URI;
import java.net.URL;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.ConcurrentHashMap;

/**
 * RegistryHandler
 *
 * @author Lss
 * @date 2020/2/18 23:58
 * @Version 1.0
 */
public class RegistryHandler extends ChannelInboundHandlerAdapter {

    //Save all available services with
    public static ConcurrentHashMap<String, Object> registryMap = new ConcurrentHashMap<String,Object>();

    //Save all related service classes
    private List<String> classNames = new ArrayList<String>();

    public RegistryHandler(){
        //Complete recursive scan
        scannerClass("com.xinfan.netty.rpc.provider");
        doRegister();
    }


    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
        Object result = new Object();
        InvokerProtocol request = (InvokerProtocol)msg;

        //When the client establishes a connection, it needs to obtain information from the custom protocol and get specific services and parameters
        //Using reflection calls
        if(registryMap.containsKey(request.getClassName())){
            Object clazz = registryMap.get(request.getClassName());
            Method method = clazz.getClass().getMethod(request.getMethodName(), request.getParames());
            result = method.invoke(clazz, request.getValues());
        }
        ctx.write(result);
        ctx.flush();
        ctx.close();
    }

    @Override
    public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
        cause.printStackTrace();
        ctx.close();
    }


    /*
     * Recursive scanning
     */
    private void scannerClass(String packageName){
        URL url = this.getClass().getClassLoader().getResource(packageName.replaceAll("\\.", "/"));
        File dir = new File(url.getFile());
        for (File file : dir.listFiles()) {
            //If it is a folder, continue recursion
            if(file.isDirectory()){
                scannerClass(packageName + "." + file.getName());
            }else{
                classNames.add(packageName + "." + file.getName().replace(".class", "").trim());
            }
        }
    }

    /**
     * Completion of registration
     */
    private void doRegister(){
        if(classNames.size() == 0){ return; }
        for (String className : classNames) {
            try {
                Class<?> clazz = Class.forName(className);
                Class<?> i = clazz.getInterfaces()[0];
                registryMap.put(i.getName(), clazz.newInstance());
            } catch (Exception e) {
                e.printStackTrace();
            }
            }
        }
}

At this point, the basic functions of the registry have been completed. Let's see the code implementation of the client.

3.5 implementation of Consumer remote call

Sort out the basic implementation ideas, mainly to complete a function like this: the interface function in the API module is implemented in the server (not in the client). Therefore, when a client calls an interface method defined in the API, it actually needs to initiate a network request to call a service on the server. The network request is first received by the registration center. The registration center first determines the location of the service to be called, then forwards the request to the real service implementation, finally calls the server code, and transmits the return value to the client through the network. The whole process is completely insensitive to the client, just like calling a local method. The specific calling process is shown in the following figure:

Let's look at the code implementation and create the RpcProxy class:

package com.xinfan.netty.rpc.consumer.proxy;

import com.xinfan.netty.rpc.protocol.InvokerProtocol;
import io.netty.bootstrap.Bootstrap;
import io.netty.channel.*;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioSocketChannel;
import io.netty.handler.codec.LengthFieldBasedFrameDecoder;
import io.netty.handler.codec.LengthFieldPrepender;
import io.netty.handler.codec.serialization.ClassResolvers;
import io.netty.handler.codec.serialization.ObjectDecoder;
import io.netty.handler.codec.serialization.ObjectEncoder;

import java.lang.reflect.InvocationHandler;
import java.lang.reflect.Method;
import java.lang.reflect.Proxy;

/**
 * RpcProxy
 *
 * @author Lss
 * @date 2020/2/19 12:10
 * @Version 1.0
 */
public class RpcProxy  {

    public static <T> T create(Class<?> clazz){
        //clazz came in as an interface
        MethodProxy proxy = new MethodProxy(clazz);
        Class<?> [] interfaces = clazz.isInterface() ?
                new Class[]{clazz} :
                clazz.getInterfaces();
        T result = (T) Proxy.newProxyInstance(clazz.getClassLoader(),interfaces,proxy);
        return result;
    }

    private static class MethodProxy implements InvocationHandler {
        private Class<?> clazz;
        public MethodProxy(Class<?> clazz){
            this.clazz = clazz;
        }

        @Override
        public Object invoke(Object proxy, Method method, Object[] args)  throws Throwable {
            //If the passed in is an implemented concrete class (this demonstration skips this logic)
            if (Object.class.equals(method.getDeclaringClass())) {
                try {
                    return method.invoke(this, args);
                } catch (Throwable t) {
                    t.printStackTrace();
                }
                //If you pass in an interface (core)
            } else {
                return rpcInvoke(proxy,method, args);
            }
            return null;
        }


        /**
         * The core method of implementing interface
         * @param method
         * @param args
         * @return
         */
        public Object rpcInvoke(Object proxy,Method method,Object[] args){

            //Transport protocol encapsulation
            InvokerProtocol msg = new InvokerProtocol();
            msg.setClassName(this.clazz.getName());
            msg.setMethodName(method.getName());
            msg.setValues(args);
            msg.setParames(method.getParameterTypes());

            final RpcProxyHandler consumerHandler = new RpcProxyHandler();
            EventLoopGroup group = new NioEventLoopGroup();
            try {
                Bootstrap b = new Bootstrap();
                b.group(group)
                        .channel(NioSocketChannel.class)
                        .option(ChannelOption.TCP_NODELAY, true)
                        .handler(new ChannelInitializer<SocketChannel>() {
                            @Override
                            public void initChannel(SocketChannel ch) throws Exception {
                                ChannelPipeline pipeline = ch.pipeline();
                                //Custom protocol decoder (the code of the server can be copied)
                                /** There are 5 input parameters, which are explained as follows
                                 maxFrameLength: The maximum length of the frame. If the length of the frame is greater than this value, a TooLongFrameException is thrown.
                                 lengthFieldOffset: Offset of length field: the position of the corresponding length field in the whole message data
                                 lengthFieldLength: Length field length: for example, if the length field is int type, then the value is 4 (long type is 8)
                                 lengthAdjustment: Compensation value to add to length field value
                                 initialBytesToStrip: Number of first bytes removed from decoded frame
                                 */
                                pipeline.addLast("frameDecoder", new LengthFieldBasedFrameDecoder(Integer.MAX_VALUE, 0, 4, 0, 4));
                                //Custom protocol encoder
                                pipeline.addLast("frameEncoder", new LengthFieldPrepender(4));
                                //Object parameter type encoder
                                pipeline.addLast("encoder", new ObjectEncoder());
                                //Object parameter type decoder
                                pipeline.addLast("decoder", new ObjectDecoder(Integer.MAX_VALUE, ClassResolvers.cacheDisabled(null)));
                                pipeline.addLast("handler",consumerHandler);
                            }
                        });

                ChannelFuture future = b.connect("localhost", 8080).sync();
                future.channel().writeAndFlush(msg).sync();
                future.channel().closeFuture().sync();
            } catch(Exception e){
                e.printStackTrace();
            }finally {
                group.shutdownGracefully();
            }
            return consumerHandler.getResponse();
        }

    }
}

Receive the return value of the network call RpcProxyHandler class:

package com.xinfan.netty.rpc.consumer.proxy;

import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelInboundHandlerAdapter;

/**
 * RpcProxyHandler
 *
 * @author Lss
 * @date 2020/2/19 12:36
 * @Version 1.0
 */
public class RpcProxyHandler extends ChannelInboundHandlerAdapter{

    private Object response;

    public Object getResponse() {
        return response;
    }

    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
        response=msg;
    }

    @Override
    public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
        System.out.println("client exception is general");
    }
}

Complete the client call code RpcConsumer class:

package com.xinfan.netty.rpc.consumer;

import com.xinfan.netty.rpc.api.IRpcHelloService;
import com.xinfan.netty.rpc.api.IRpcService;
import com.xinfan.netty.rpc.consumer.proxy.RpcProxy;

/**
 * RpcConsumer
 *
 * @author Lss
 * @date 2020/2/19 12:55
 * @Version 1.0
 */
public class RpcConsumer {

    public static void main(String[] args) {
        IRpcHelloService rpcHello= RpcProxy.create(IRpcHelloService.class);
        System.out.println(rpcHello.hello("Sichuan lover"));
        IRpcService service=RpcProxy.create(IRpcService.class);
        System.out.println("8 + 2 = " + service.add(8, 2));
        System.out.println("8 - 2 = " + service.sub(8, 2));
        System.out.println("8 * 2 = " + service.mult(8, 2));
        System.out.println("8 / 2 = " + service.div(8, 2));
        //Monitor in Dubbo is implemented with Spring's AOP embedded point. I did not introduce the Spring framework
    }
}

3.6 Monitor monitoring

The Monitor in Dubbo is implemented with Spring's AOP embedded point. I didn't introduce the Spring framework. I don't implement the monitoring function in this code. Interested partners can review the previous Spring AOP courses to improve this function.

04 operation effect demonstration

The first step is to start the registration center. The operation results are as follows:

Step 2: run the client, and the result is as follows:

Through the above case demonstration, I believe that my friends have been quite impressed by the application of Netty. This time, I just made a simple implementation of the basic implementation principle of RPC. Interested partners can continue to improve other details of RPC on the basis of this project. Welcome message