dubbo system learning

dubbo is a distributed service framework, which is committed to improving performance and transparency of RPC remote service invocation scheme and SOA Service governance scheme.

soa idea: Service Oriented Architecture

Expose the corresponding ip and port to each module and run it as a service

Focus on Service Management (load balancing, disaster recovery mode, horizontal expansion of services)

2. Evolution of Architecture

With the development of the Internet and the continuous expansion of the scale of website applications, the conventional vertical application architecture can not cope with it. The distributed service architecture and mobile computing architecture are imperative. A governance system is urgently needed to ensure the orderly evolution of the architecture.

Single application architecture
 When the website traffic is very small, only one application is needed to deploy all functions together to reduce the deployment nodes and costs. At this time, the data access framework (ORM) used to simplify the workload of addition, deletion, modification and query is the key.

Vertical application architecture
 When the number of visits increases gradually, the acceleration brought by the increase of a single application machine becomes smaller and smaller. One way to improve efficiency is to split the application into several unrelated applications to improve efficiency. At this point, the Web framework (MVC) for accelerating front-end page development is the key.

Distributed service architecture
 When there are more and more vertical applications, the interaction between applications is inevitable. Extract the core business as an independent service, and gradually form a stable service center, so that the front-end applications can respond to the changing market demand more quickly. At this time, the distributed service framework (RPC) for improving business reuse and integration is the key.

Mobile computing architecture (think this is a micro service)
When there are more and more services, problems such as capacity evaluation and waste of small service resources gradually appear. At this time, it is necessary to add a dispatching center to manage the cluster capacity in real time based on the access pressure and improve the cluster utilization. At this time, the resource scheduling and Governance Center (SOA) for improving machine utilization is the key.

Distributed service architecture:

3. Architecture diagram of Dubbo( link)

node	Role description
Provider	Service provider of exposed services
Consumer	The service consumer that invokes the remote service
Registry	Registry for service registration and discovery
Monitor	The monitoring center that counts the number and time of service calls
Container	Service running container, spring is recommended

1.dubbo demo( link)

1. Service providers

1. Introduce the dependency spring dubbo zookeeper zkclient

  <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.12</version>
        </dependency>

        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-core</artifactId>
            <version>4.3.4.RELEASE</version>
        </dependency>

        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-context</artifactId>
            <version>4.3.4.RELEASE</version>
        </dependency>

        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-beans</artifactId>
            <version>4.3.4.RELEASE</version>
        </dependency>

        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>dubbo</artifactId>
            <version>2.5.3</version>
        </dependency>

        <dependency>
            <groupId>org.apache.zookeeper</groupId>
            <artifactId>zookeeper</artifactId>
            <version>3.4.12</version>
        </dependency>

        <dependency>
            <groupId>com.101tec</groupId>
            <artifactId>zkclient</artifactId>
            <version>0.10</version>
        </dependency>

2. Develop service classes (interface and implementation classes)

public interface DemoService {
    String sayHello(String name);
}
public class DemoServiceImpl implements DemoService {
    @Override
    public String sayHello(String name) {
        System.out.println("hello"+name);
        return "hello"+name;
    }
}

3.provider.xml

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:dubbo="http://code.alibabatech.com/schema/dubbo"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://code.alibabatech.com/schema/dubbo http://code.alibabatech.com/schema/dubbo/dubbo.xsd">
    <!-- Provider application information for calculating dependencies       Unique application name, identification (project module name)-->
    <dubbo:application name="dubbo_provider1"  />

    <!-- use multicast Broadcast registry service address multicast://224.5.6.7:1234 designated Registry (zookeeper) - >
    <dubbo:registry address="zookeeper://192.168.231.135:2181" />

    <!-- use dubbo The protocol exposes services on port 20880     Agreement for the specified service: dubbo Protocol (fixed) and port number used (optional)-->
    <dubbo:protocol name="dubbo" port="20880" />

    <!-- Declare the service interfaces that need to be exposed       Register service to registry-->
    <dubbo:service interface="com.liziyi.service.DemoService" ref="demoService" />

    <!-- And local bean Implement services like    Service implementation class-->
    <bean id="demoService" class="com.liziyi.service.impl.DemoServiceImpl" />
</beans>

4. Publishing services

public class Provider {
    public static void main(String[] args) throws Exception {
        ClassPathXmlApplicationContext context = new ClassPathXmlApplicationContext("provider.xml");
        context.start();
        System.out.println("Service provider, start providing services----------------");
        System.in.read(); // press any key to exit
    }
}

2. Create service consumers

1. Introduce the dependency spring dubbo zookeeper zkclient

2. Call service configuration service

1. Copy the producer interface to the consumer

public interface DemoService {
    String sayHello(String name);
}

3.consumer.xml

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:dubbo="http://code.alibabatech.com/schema/dubbo"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://code.alibabatech.com/schema/dubbo http://code.alibabatech.com/schema/dubbo/dubbo.xsd">

    <!-- The consumer's application name is used to calculate dependencies. It is not a matching condition. It should not serve consumers like the provider-->
    <dubbo:application name="dubbo_consumer1"  />

    <!-- use multicast Broadcast registry exposure discovery service address     Configure registry(zookeeper)-->
    <dubbo:registry address="zookeeper://192.168.231.135:2181" />

    <!-- Generate a remote service proxy, which can communicate with local bean Same use demoService    Call service-->
    <dubbo:reference id="demoService" interface="com.liziyi.service.DemoService" />
</beans>

3. Service call

public class Consumer {
    public static void main(String[] args) throws Exception {
        ClassPathXmlApplicationContext context = new ClassPathXmlApplicationContext("consumer.xml");
        context.start();
        DemoService demoService = (DemoService)context.getBean("demoService"); // Get remote service proxy
        String hello = demoService.sayHello("world"); // Execute remote method
        System.out.println( hello ); // Display call results
    }
}

3. Connect zookeeper (service registration and discovery Registry)

Installing zookeeper

cd /opt/software/zookeeper/bin
#Connect client
./zkCli.sh

This proves that the connection is successful!

Check the contents

dubbo%3A%2F%2F192.168.231.1%3A20880%2Fcom.liziyi.service.DemoService%3Fanyhost%3Dtrue%26application%3Ddubbo_provider1%26dubbo%3D2.5.3%26interface%3Dcom.liziyi.service.DemoService%26methods%3DsayHel%3D504%26side%3Dprovider%26timestamp%3D1626694429185

Transcoding:

public class TestDubborProviders {
    @Test
    public void test() throws UnsupportedEncodingException {
        String s = "dubbo%3A%2F%2F192.168.231.1%3A20880%2Fcom.liziyi.service.DemoService%3Fanyhost%3Dtrue%26application%" +
                "3Ddubbo_provider1%26dubbo%3D2.5.3%26interface%3Dcom.liziyi.service.DemoService%26methods%3DsayHel%3D504" +
                "%26side%3Dprovider%26timestamp%3D1626694429185";
        String decode = URLDecoder.decode(s, "UTF-8");
        System.out.println(decode);
    }
}

result:

dubbo://192.168.231.1:20880/com.liziyi.service.DemoService?anyhost=true&application=dubbo_provider1&dubbo=2.5.3&interface=com.liziyi.service.DemoService&methods=sayHel=504&side=provider&timestamp=1626694429185

2. Three development modes of Dubbo

1. demo above Dubbo + spring

2.dubbo original API

3.dubbo+spring annotated development

3. Configuration priority of Dubbo

Set expiration time:( link)

Method level takes precedence, interface level takes precedence, and global configuration takes precedence.

If the level is the same, the consumer takes priority, followed by the provider.

(it is recommended that the service provider set the timeout, because the service provider knows how long a method needs to be executed. If a consumer references multiple services at the same time, it does not need to care about the timeout setting of each service).

4. Attribute configuration( 🔗)

5. Annotation configuration( 🔗)

Using @ Service does not need to be in the provider Specify the following items in the XML file:

@Service
public class AnnotationServiceImpl implements AnnotationService {
    @Override
    public String sayHello(String name) {
        return "annotation: hello, " + name;
    }
}

4.dubbo usage

1. Check at startup( 🔗)

2. Cluster fault tolerance( 🔗)

1. Cluster fault tolerance of services:

1. Set up 3 or more services of service providers

When making a cluster, copy duboo_ If the following occurs after provider1:

rename can be renamed

2. Start multiple services

Because it is on one machine, three ports 20881, 2 and 3 are configured

3. Cluster load balancing strategy( 🔗)

It is recommended to configure on the server side

Random LoadBalance
Random, set random probability by weight.
The probability of collision on a section is high, but the larger the adjustment amount is, the more uniform the distribution is, and the weight is relatively uniform after being used according to the probability, which is conducive to the dynamic adjustment of the provider's weight.
RoundRobin LoadBalance
Polling: set the polling ratio according to the weight after the Convention.
There is a problem that slow providers accumulate requests. For example, the second machine is very slow but does not hang up. When the request is transferred to the second machine, it gets stuck there. Over time, all requests get stuck in the second machine.
LeastActive LoadBalance
The minimum number of active calls is the random number of the same active number. The active number refers to the count difference before and after the call.
Make slow providers receive fewer requests, because the slower the provider, the greater the count difference before and after the call.
ConsistentHash LoadBalance
Consistency Hash: requests with the same parameters are always sent to the same provider.
When a provider hangs up, the requests originally sent to the provider are shared among other providers based on the virtual node, without causing drastic changes.
For the algorithm, see: http://en.wikipedia.org/wiki/Consistent_hashing
Only the first parameter Hash is used by default. If you want to modify it, configure < Dubbo: parameter key = "Hash. Arguments" value = "0,1" / >
160 virtual nodes are used by default. If you want to modify them, please configure < Dubbo: parameter key = "hash. Nodes" value = "320" / >
match

Consistent hash algorithm

Because of hot issues, virtual nodes are proposed.

By default, 160 copies are distributed evenly on the circle. Finding a virtual node is actually the corresponding real node.

Real node:

192.168.0.3

Virtual: 1.1.1.0-1.1.1.159 (example)

192.168.0.4

Virtual: 2.1.1.0-2.1.1.159

192.168.0.5

Virtual: 3.1.1.0-3.1.1.159

4. Cluster fault tolerance( 🔗)

5. Thread model( 🔗)

If the logic of event processing can be completed quickly and new IO requests will not be initiated, such as just recording an ID in memory, it is faster to process directly on the IO thread because the thread pool scheduling is reduced.

However, if the event processing logic is slow or a new IO request needs to be initiated, such as querying the database, it must be sent to the thread pool, otherwise the IO thread will be blocked and other requests will not be received.

If an IO thread is used to process events and a new IO request is initiated during the event processing process, such as a login request in a connection event, an exception of "possible deadlock" will be reported, but the deadlock will not be true.

See the official website for parameter details

6. Direct provider( 🔗)

In the development and testing environment, it is often necessary to bypass the registration center and only test the specified service providers. At this time, point-to-point direct connection may be required. The point-to-point direct connection mode will take the service interface as the unit and ignore the provider list of the registration center. The point-to-point configuration of interface A will not affect interface B to obtain the list from the registration center.

Usage: on the consumer side

<dubbo:reference id="xxxService" interface="com.alibaba.xxx.XxxService" url="dubbo://localhost:20890" />

7. Multi protocol( link)

1.dubbo agreement( link)

Dubbo default protocol adopts single long connection and NIO asynchronous communication, which is suitable for small amount of data and large concurrent service calls, and the number of service consumer machines is much larger than that of service provider machines.

On the contrary, the Dubbo default protocol is not suitable for transmitting services with a large amount of data, such as files and videos, unless the request volume is very low.

2.rmi protocol

Scope of application: the size of incoming and outgoing parameter packets is mixed, the number of consumers and providers is about the same, and files can be transferred.
Applicable scenario: routine remote service method call, interoperation with native RMI service

8. Multi registry( 🔗)

Dubbo supports the simultaneous registration of the same service with multiple registries, or the registration of different services in different registries, or even the simultaneous reference of services with the same name registered in different registries. In addition, the registry supports custom extensions

9. Service grouping( 🔗)

10.JSON generalization call( 🔗)

For Dubbo generalized calls, a new method is provided: passing a string directly to complete a call. That is, the user can directly pass the json string of the parameter object to complete a Dubbo generalization call

When the client has no interface, how to call the service of the server?

Client's consumer Add generic="true" on XML

    <dubbo:reference id="demoService" interface="com.liziyi.service.DemoService" generic="true" />

client

public class Consumer {
    public static void main(String[] args) throws Exception {
        ClassPathXmlApplicationContext context = new ClassPathXmlApplicationContext("consumer.xml");
        GenericService genericService = (GenericService) context.getBean("demoService");
        // Pass the json string of the parameter object for a call
            // Parameter 1: called method name
            // Parameter 2: call parameter type
            // Parameter 3: call the value corresponding to the parameter type
        Object res = genericService.$invoke("sayHello", new String[]{"java.lang.String"}, new Object[]{"xiaohei"});
        System.out.println("result[setUser]: "+res); // Response result: result[setUser]: {name=Tom, class=com.xxx.api.service.User, age=24}
        System.in.read(); // press any key to exit
    }
}

11. Best practices of service( 🔗)

edition:

Each interface shall define a version number to make it possible for subsequent incompatible upgrades, such as < Dubbo: service interface = "com. XXX. Xxxservice" version = "1.0" / >.

It is recommended to use a two digit version number, because the third digit version number usually indicates a compatible upgrade. The service version needs to be changed only when it is incompatible.

In case of incompatibility, first upgrade half of the providers to the new version, then upgrade all consumers to the new version, and then upgrade the remaining half of the providers to the new version.

call

The Provider side needs to verify the input parameters. For performance considerations, the service implementer can consider adding a service Stub class to the API package to complete the inspection. The following parameters are verified:

Parameter validation( 🔗)

5. Integration of springboot and dubbo

1.springboot and dubbo dependency

 <!--dubbo-springBoot rely on springboot Yes 1.5.7 Only use dubbo Of 1.0.2 1.5.6 Corresponding 1.0.1  2.1.5 Corresponding 1.1.3-->
        <dependency>
            <groupId>com.gitee.reger</groupId>
            <artifactId>spring-boot-starter-dubbo</artifactId>
            <version>1.0.2</version>
        </dependency>


        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.12</version>
        </dependency>
        <!--integration web development-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <!--testclient-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
        </dependency>

        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <version>1.16.18</version>
        </dependency>

        <dependency>
            <groupId>org.apache.tomcat.embed</groupId>
            <artifactId>tomcat-embed-jasper</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-tomcat</artifactId>
        </dependency>

        <dependency>
            <groupId>jstl</groupId>
            <artifactId>jstl</artifactId>
            <version>1.2</version>
        </dependency>
        <!--mybatis-->
        <dependency>
            <groupId>org.mybatis.spring.boot</groupId>
            <artifactId>mybatis-spring-boot-starter</artifactId>
            <version>1.3.2</version>
        </dependency>

        <dependency>
            <groupId>commons-dbcp</groupId>
            <artifactId>commons-dbcp</artifactId>
            <version>1.4</version>
        </dependency>

        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <version>5.1.46</version>
        </dependency>

2. Table building

3. Split the module

cmfz_ user cmfz_ commons cmfz_ book cmfz_ back_ Sys (background) cmfz_ front_ Sys (front desk)

4. Develop user services

In this way, the child module can get all the dependencies of the parent module.

1). The business layer adds the third annotation below

@Service
@Transactional
@com.alibaba.dubbo.config.annotation.Service(interfaceClass = UserService.class)
public class UserServiceImpl implements UserService {

2). Configure dubbo in the configuration file

server:
  port: 8989
spring:
  datasource:
    url: jdbc:mysql://localhost:3306/test
    data-username: root
    data-password: 123456
    driver-class-name: com.mysql.jdbc.Driver
  dubbo:
    application:
      name: cmfz_user
    registry:
      address: zookeeper://192.168.231.136:2181
    protocol:
      name: dubbo
      port: 20881
    base-package: com.liziyi.cmfz.service
mybatis:
  mapper-locations: classpath:com/liziyi/mapper/*.xml

Topics: Dubbo

Programmer Think