First of all, this is not a code-based article. It is suitable for readers who want to have a better understanding of dubbo. The article is not too deep, but a series of points of knowledge are colluded to help readers learn something new from the past.

Introduction to RPC Services

Readers with some distributed development experience are confident that they have used some RPC frameworks and that calling remote services through API interfaces provided after the frameworks are wrapped feels as easy as calling local services.How can such a convenient and useful technical framework be packaged in the actual development process?

Earlier, foreign engineers designed a technology that can invoke applications on computer B through computer A. This technology does not require developers to know much about network communication, and it is as convenient to invoke programs on other machines as calling local programs.

Machine A initiates a request to invoke the program of Machine B, which is suspended. Machine B receives the parameters of the request initiated by Machine A and then performs some parameter conversion. The result of the corresponding program is returned to A, which is the original RPC service call.

Advantages of RPC calls

simple

There is no need for developers to make too many settings for network communication, such as writing more http protocol parameters (header, context, Accept-Language,Accept-Encode, etc.) when making remote interface calls using http protocol, which are not actually very friendly for developers.However, RPC service invocation frameworks usually encapsulate such parsing, which greatly reduces the difficulty for developers.

Efficient

In terms of network transmission, RPC is more between application layer and transport layer.Here we need to clear up a problem, network hierarchy.RPCs are part of the session layer and are lighter to invoke than HTTP in the application layer.

Common Remote Call Technologies

rmi

Implemented using the java.rmi package, based on the Java Remote Method Protocol and native serialization of java.

Hessian

Is a lightweight remoting onhttp tool that provides RMI functionality in a simple way.Based on the HTTP protocol, binary codec is used.

protobuf-rpc-pro

Is a Java class library that provides a framework for remote method calls based on Google's Proocol Buffers protocol.NIO technology based on Netty bottom layer.Supports TCP reuse / keep-alive, SSL encryption, RPC call cancellation, embedded logging and other functions.

Thrift

It is a scalable software framework for cross-language services.It has a powerful code generation engine that seamlessly supports C++, C#, Java, Python, and PHP and Ruby.thrift allows you to define a description file that describes data types and service interfaces.Based on this file, the compiler can easily generate RPC client and server communication code.

Originally developed by facebook as RPC communication between the internal languages of the system, it contributed to the Apache fund in 2007 and is now one of the opensource s under apache.Supports RPC-style communication between multiple languages: php language client s can construct an object, invoke corresponding service methods to invoke services in the java language, and C/S RPC calls across languages.The underlying communication is based on SOCKET.

Avro

From Doug Cutting, Hadoop's father, Avro's goal with Thrift's popularity is not only to provide a set of Thrift-like communication middleware, but also to create a new, standard Protocol for data exchange and storage in cloud computing.Supports both HTTP and TCP protocols.

Dubbo

Dubbo is an excellent open source service framework for Alibaba. It enables applications to implement service output and input functions through high performance RPC s, which can be seamlessly integrated with the Spring framework.

We talked about the history of RPC remote calls above, so let's talk more about RPC services below.

First let's look at the network protocol content for OSI.

Seven-Layer Network Model for OSI

For the seven-layer network model of OSI, I draw the following diagram:

Below is my personal understanding of the seven-tier agreement:

• The application tier mainly defines the format of the service interface, such as providing a certain terminal interface to expose to external application calls.

• Representation layer handles format conversion of some data transfers, such as uniform encoding, encryption and decryption.

• Session Layer manages user sessions and dialogs and establishes session connections between different machines.

• Transport Layer provides reliable and ordered packet information to the network layer.

• The network layer, the layer that actually sends packet information, provides flow and congestion control to reduce network resource consumption.

• Data Link Layer encapsulates corresponding data packets to detect and correct data packet transmission information.

• Physical Layer Sends Data Through Network Communication Devices

HTTP & RPC

HTTP is mainly located in the application layer part of the TCP/IP protocol stack. First, you need to build a link that shakes hands three times before you can send a request for data information and then break the link with a wave four times.

RPC crosses the transport and application layers in the request process because it is dependent on the Socket itself.(I don't know the reason for this).Reducing the top layer encapsulation naturally makes RPC requests much more efficient than HTTP.

So what parts should a complete RPC call contain?

Usually we divide a complete RPC architecture into the following core components:

• Server

• Client

• Server Stub

• Client Stub

Let me talk a little about stub in these four modules.This word is translated as a stub.

Client Stub is to package the parameters, service name and service address requested by the client and send them to the server side uniformly.

Server Stub I'll explain in a plain language that the server unpacks the message and calls the local method after it receives the data sent by the Client.(You should have seen the netty unpacking mechanism to understand this).

Core properties of Dubbo

In fact, the core content of Dubbo configuration is service exposure, service discovery, and service governance.

What are service exposure, service discovery, service governance?

Below we will explain with a configuration of xml:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:dubbo="http://code.alibabatech.com/schema/dubbo"
       xmlns="http://www.springframework.org/schema/beans"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
    http://code.alibabatech.com/schema/dubbo http://code.alibabatech.com/schema/dubbo/dubbo.xsd">
    <dubbo:application name="dubbo-invoker-provider">
        <dubbo:parameter key="qos.port" value="22222"/>
    </dubbo:application>
    <dubbo:registry address="zookeeper://127.0.0.1:2181"/>
    <dubbo:protocol name="dubbo" port="20880"/>
    <bean id="userService" class="com.sise.user.service.UserServiceImpl" />
    <dubbo:service interface="com.sise.user.service.UserService" ref="userService" />
</beans>

 

In the Dubbo configuration file, what we usually refer to as dubbo:service is service exposure, dubbo:refernce is service discovery, mock is service governance, timeout is one of service governance (performance tuning).

Assuming that dubbo wants to extract some of the public configurations, we can configure them through the properties file, which has the following priority in loading the configuration file:

1. Priority will read after JVM-D startup parameters

2. Read xml configuration file

3. Read property profile content

By default, Dubbo reads information from the dubbo.properties configuration file, such as the following:

dubbo.application.name=dubbo-user-service
dubbo.registry.address=zookeeper://127.0.0.1:2181

 

Assuming our Dubbo configuration file is not named dubbo.properties (assuming it is named my-dubbo.properties), you can add this instruction after the boot parameter:

 -Ddubbo.properties.file=my-dubbo.properties

 

After the application starts, the corresponding project reads the specified configuration file so that some shared dubbo configurations can be extracted.

Mapping of XML and Configuration Classes

In our work, we usually configure xml to set a service interface exposed by the service side and the service information that the consumer needs to invoke. These configurable xml are actually parsed into the corresponding entity class object in the source code of dubbo.

For example, the reference configuration class that we commonly use, I paste a code below:

package com.sise.user.config;
import com.sise.user.service.UserService;
import com.sise.user.service.UserServiceImpl;
import org.apache.dubbo.config.*;
import java.io.IOException;
import java.util.concurrent.CountDownLatch;
/**
 * dubbo Custom Configuration Class Inside
 *
 * @author idea
 * @data 2019/12/29
 */
public class DubboSelfDefConfig {
    /**
     * dubbo Service Exposure
     */
    public void server() {
        ApplicationConfig applicationConfig = new ApplicationConfig();
        applicationConfig.setName("dubbo-server-config");
        RegistryConfig registryConfig = new RegistryConfig();
        registryConfig.setAddress("zookeeper://127.0.0.1:2181");
        ProtocolConfig protocolConfig = new ProtocolConfig();
        protocolConfig.setName("dubbo");
        protocolConfig.setPort(20880);
        protocolConfig.setThreads(200);
        UserService userService = new UserServiceImpl();
        ServiceConfig<UserService> serviceConfig = new ServiceConfig<>();
        serviceConfig.setApplication(applicationConfig);
        serviceConfig.setRegistry(registryConfig);
        serviceConfig.setProtocol(protocolConfig);
        serviceConfig.setInterface(UserService.class);
        serviceConfig.setRef(userService);
        serviceConfig.export();
    }

    public void consumer() {
        ApplicationConfig applicationConfig = new ApplicationConfig();
        applicationConfig.setName("dubbo-client-config");
        RegistryConfig registryConfig = new RegistryConfig();
        registryConfig.setAddress("zookeeper://127.0.0.1:2181");
        ReferenceConfig<UserService> referenceConfig = new ReferenceConfig<>();
        referenceConfig.setApplication(applicationConfig);
        referenceConfig.setRegistry(registryConfig);
        referenceConfig.setInterface(UserService.class);
        UserService localRef = referenceConfig.get();
        localRef.echo("idea");
    }
    public static void main(String[] args) throws InterruptedException, IOException {
        DubboSelfDefConfig d = new DubboSelfDefConfig();
        d.consumer();
        CountDownLatch countDownLatch = new CountDownLatch(1);
        countDownLatch.await();
    }
}

 

Inside this code, you can find this information by case:

    UserService localRef = referenceConfig.get();
    localRef.echo("idea");

 

These two lines of statements are at the heart of getting a specific service, and since I have defined a public service interface called UserService elsewhere, I can convert it during a service reference.

Three Big New Trends for Dubbo 2.7

Dubbo's official GitHub address is https://github.com/apache/dubbo

Insert a picture description here

Dubbo currently has five branches as shown in the figure, of which 2.7.1-release is only a temporary branch, negligible. For the other four branches, I summarize the following information:

 

• 2.5.x has recently passed a vote and the Dubbo community is about to stop maintaining it.

• 2.6.x is a long-term supported version and is a pre-Apache version contributed by Dubbo with a package name prefix of com.alibaba and a JDK version of 1.6.

• 3.x-dev is a forward-looking version that adds some advanced features to Dubbo, such as support for the rx feature.

• master is a long-term supported version with version number 2.7.x and is a development version contributed by Dubbo to Apache with package name prefix: org.apache and JDK version 1.8.

Dubbo 2.7 New Features

As an incubator version of Apache, Dubbo 2.7.x adds many new features that weigh heavily in addition to code optimization. This article will introduce two of the most typical new features:

• Asynchronous transformation

• Renovation of three major centers

Asynchronous transformation

1. Asynchronous invocation method, which provides the function of asynchronous invocation in Dubbo version 2.7, has the following case code:

@RestController
@RequestMapping(value = "/test")
public class TestController {
    @Reference(async = true)
    private UserService userService;

    @GetMapping("/testStr")
    public String testStr(String param){
        return userService.testEcho(param);
    }
}

 

But we usually don't get the response value by sending it asynchronously, so the return here is null.

If you want to get the response value returned asynchronously in a dubbo framework less than version 2.7, you still need to extract information through the RPC context.

The code case is as follows:

 @GetMapping("/futureGet")
    public String futureGet(String param) throws ExecutionException, InterruptedException {
        userService.testEcho(param);
        Future<String> future= RpcContext.getContext().getFuture();
        String result = future.get();
        System.out.println("this is :"+result);
        return result;
    }

 

Response values can be retrieved in the RPC context, but this method requires some waiting, so efficiency will be reduced.Suppose we upgrade the dubbo version to 2.7.1 and use CompletableFuture for interface optimization, this part of the code implementation will change:

/**
 * @author idea
 * @date 2019/12/31
 * @Version V1.0
 */
public interface DemoService {
      String sayHello(String name) ;
      default CompletableFuture<String> sayAsyncHello(String name){
            return CompletableFuture.completedFuture(sayHello(name));
      }
}

 

Caller code:

package com.sise.consumer.controller;

import com.sise.dubbo.service.DemoService;
import org.apache.dubbo.config.annotation.Reference;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.atomic.AtomicReference;
/**
 * @author idea
 * @date 2019/12/31
 * @Version V1.0
 */
@RestController
@RequestMapping(value = "/demo")
public class DemoController {

    @Reference
    private DemoService demoService;

    @RequestMapping(value = "/testDemo")
    public String testDemo(String name){
        System.out.println("[testDemo] this is :"+name);
        return demoService.sayHello(name);
    }.

    @RequestMapping(value = "/testAsyncDemo")
    public String testAsyncDemo(String name){
        System.out.println("[testAsyncDemo] this is :"+name);
        CompletableFuture<String> future = demoService.sayAsyncHello(name);
        AtomicReference<String> result = null;
        //Processing data information for responses through a callback thread
        future.whenComplete((retValue,exception)->{
           if(exception==null){
               System.out.println(retValue);
               result.set(retValue);
           } else {
               exception.printStackTrace();
           }
        });
        return "Through one callback Threads to process response data information,So we can't get a response at this time";
    }
}

 

The callback thread was used to help us with the original data content. I borrowed an official diagram to show the asynchronous calls in dubbo:

Many of the methods we've explained above are just for dubbo's client asynchronization, not for service-side asynchronization, because thinking in conjunction with dubbo's business thread pool model, service-side asynchronization is more expensive (because the thread pool within Dubbo is itself an asynchronous call).

Of course, the configuration for interface asynchronous calls in dubbo 2.6 is still valid at version 2.7.

Renovation of Three Centers

Registration Center

Prior to Dubbo 2.7, Dubbo was mainly composed of consumer s, provider s, register s. However, after version 2.7, dubbo's registration center was broken down into three centers, the original registration center, the metadata center, and the configuration center.

Metadata Configuration

In Dubbo version 2.7, too much data that was originally registered on the zk side was registered and split to ensure less pressure on the zk side.The configuration is as follows:

<dubbo:registry address="zookeeper://127.0.0.1:2181" simplified="true"/>

 

After simplifying the configuration, dubbo will only upload some necessary service governance data, leaving only the following information for the simplified version of service data:

dubbo://30.5.120.185:20880/com.sise.TestService?
application=test-provider&
dubbo=2.0.2&
release=2.7.0&
timestamp=1554982201973

 

For other metadata information, it will be stored in some metadata centers, such as redis, nacos, zk, etc.

The main problems solved by metadata configuration transformation are: large push - > large storage - > large network transmission - > serious delay

Configuration Center

Dubbo 2.7 has started to support components from a variety of distributed configuration centers.For example: zk, Spring Cloud Config, Apollo, Nacos, there is a lot of information about configuring this section on the Internet, so I won't go into details here.