Microservice design guidance - use the "dead letter delay" queue to completely solve the database "deadlock" problem during business timeout compensation

Posted by Zoofu on Fri, 04 Feb 2022 11:06:31 +0100

brief introduction

We often encounter the scenario of business timeout. For example: after the payment is requested to the payment gateway, but the payment gateway is either due to the problem of the third-party payment channel or the network, the payment callback is not "successful" or there is no callback request to notify the enterprise of the relevant payment status. In this case, our dear programmers, the design I have seen most is like to use: run a JOB every X minutes, and then change the status of all orders with status = unpaid success in the database (for subsequent business operations).

Hey... The problem lies in the running JOB. This is also a typical accident of lack of training for programmers or relevant R & D teams. Database deadlock often occurs in this case.

Imagine if there are hundreds of such jobs in a medium and large-scale project... Hey, how much your company's DBA has to collapse. No, let's take a look at an example.

Analog deadlock

preparation

We write a paragraph of mybatis dao, which will retrieve the payment timeout status with status of 1 from the database (we assume that status=1 is unresponsive) and change its status to "business compensation simulation" of "2";
Accordingly, we need a service method to call this dao method;
Use junit test case to start five threads and set the same priority to run every X seconds (in order to quickly reproduce the deadlock problem, we use 1-second timeout to simulate the payment timeout). Therefore, five threads (simulating the cluster pages of 5 payments) will update the status in the database at the same time every 1 second;
We created 8.84 million pieces of data in the payment table of the database;

PaymentDao.xml

	<update id="updatePaymentStatusByLimit">
		update sky_payment set status='2' where status='1' limit 3000
	</update>

Corresponding paymentdao Java content

 public int updatePaymentStatusByLimit();

PaymentService.java

@Resource
    private PaymentDao paymentDao;

    @Transactional(rollbackFor = Exception.class)
    public void updatePaymentStatusByLimit() throws Exception {
        try {
            int records = paymentDao.updatePaymentStatusByLimit();
            logger.info(">>>>>>updated {}", records);
        } catch (Exception e) {
            logger.error(">>>>>>updatePamentStatusByLimit error: " + e.getMessage(), e);
            throw new Exception(">>>>>>updatePamentStatusByLimit error: " + e.getMessage(), e);
        }
    }

It's very simple. There's no pressure to write code. Who can't add, delete, change and check!

Code to start 5 threads in relevant unit tests

/**
 * System project name: org mk.demo. skypayment. service UpdatePaymentStatusServiceTest. java
 * 
 * Feb 2, 2022-11:13:35 PM 2022XX Company - All Rights Reserved
 * 
 */
package org.mk.demo.skypayment.service;

import java.util.List;
import java.util.concurrent.CountDownLatch;

import javax.annotation.Resource;

import org.junit.jupiter.api.Test;
import org.mk.demo.skypayment.SkyPaymentApp;
import org.mk.demo.skypayment.vo.PaymentBean;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.boot.test.context.SpringBootTest;

import net.minidev.json.JSONArray;

/**
 * 
 * UpdatePaymentStatusServiceTest
 * 
 * 
 * Feb 2, 2022 11:13:35 PM
 * 
 * @version 1.0.0
 * 
 */

@SpringBootTest(classes = {SkyPaymentApp.class})
public class PaymentServiceTest {
    private Logger logger = LoggerFactory.getLogger(this.getClass());
    private int threadCnt = 5;
    private CountDownLatch latch = new CountDownLatch(threadCnt);

    @Resource
    private PaymentService updatePaymentStatusService;

    @Test
    public void updatePaymentStatusByLimit() throws Exception {
        for (int i = 0; i < threadCnt; i++) {
            (new Thread(new UpdatorRunner(), "JUNIT Multithreading test")).start();
        }
        latch.await();
    }

    class UpdatorRunner implements Runnable {
        @Override
        public void run() {
            logger.info(">>>>>>[Current thread ID]:" + Thread.currentThread().getId());
            try {
                while (true) {
                    updatePaymentStatusService.updatePaymentStatusByLimit();
                    Thread.sleep(1000);
                }
            } catch (Exception e) {
                logger.error(e.getMessage(), e);
            }
            latch.countDown(); // After execution, the counter is decremented by 1
        }
    }

}

8.47 million pieces of data were created in the database

CREATE TABLE `sky_payment` (
  `pay_id` int(11) NOT NULL AUTO_INCREMENT,
  `status` tinyint(3) DEFAULT NULL,
  `transf_amount` varchar(45) DEFAULT NULL,
  `created_date` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
  `updated_date` timestamp NULL DEFAULT NULL ON UPDATE CURRENT_TIMESTAMP,
  PRIMARY KEY (`pay_id`)
) ENGINE=InnoDB AUTO_INCREMENT=9098509 DEFAULT CHARSET=utf8mb4;

Then run it. What do we see?

Wow... I used two databases with 512GB memory. The CPU is a 128 core server, which is installed in the MySQL master slave read-write splitting mode. In only one minute, the following gadgets appeared...

As you know, in general, in production, the number of clusters of our single micro service is generally about 10-20 to support the daily concurrency of about 90-10000. Therefore, it does not take 5 minutes. If there is a duplicate update statement between two frames in 1-2 minutes, the whole production db will be directly stuck.

At this time, if the number of rows in your database is > one million, 100% of the data will produce "DB master-slave delay", and then "all the way up card", and then your whole mall will "Bye".

All this comes from being unfamiliar with the principle of batch update (including delete) in the database

Don't just add, delete, modify and check.

Batch update (including delete) action, especially update Where or delete Where is the of lock data. A Tomcat / netty (a spring boot application anyway) is a thread. When > 1 thread runs the update at the same time Where and at the same time, if there is data duplication in the where conditions of different update/delete, deadlock will occur.

This problem will never happen on a single machine, but only in a cluster environment.

Because in the production environment, it is impossible for us to run a single business module. If we want to deal with external traffic, we must run multiple machines and clusters to accept external traffic.

Well, your stand-alone operation ensures the correctness of the business. When you want to pop up several copies when the traffic is large, the whole mall application is instantly blocked.

This is a typical case of "not conforming to cloud native design".

It's called "thin neck and small stomach". If you can't eat it, you'll burst your stomach... Do you still let enterprises play with your design and code? If you are dissatisfied with the enterprise and the team, why use this "means" to kill the enterprise?

Hey, hey, I'm just kidding.

Let's look at the correct design

Using "delay" queue to solve the design of running batch of business compensation class

One more word:

In fact, the above design can also be changed to update Change where to restrict each update statement to be by primary key, and then string hundreds or thousands of update by IDS (depending on the performance of the database) into batch update of mybatis for processing. When processed by primary key, the database will never deadlock. But this is not consistent with cloud primordial. Because you always run batch on a single machine, it will not increase parallel computing and processing capacity as the number of applications increases.

Another point is that your job is frequency run compensation. When there is no "business need compensation", your job is actually idling, which extremely consumes system resources. This is also a very "dinosaur" design and unreasonable design.

One of the purposes of cloud native is to expand the scale of the cluster horizontally, and your system's computing and processing capacity will expand with it.

The latest version of RabbitMQ, such as 3.9, has "delay queue", and Redis also has the mature feature of delay queue.

But we still use rabbit MQ 3 8. X, because RabbitMQ used by most companies is still version 3.7-3.8. If you want to change the enterprise's underlying architecture with new features, the risk is obviously too high.

In rabbit mq3 In 8. X, there is a queue called dead letter queue, which can be used for "delay operation".

We use spring boot2 X combined with RabbitTemplate, the use of RabbitMQ's dead letter queue is configured as follows

application-local.yml

mysql:
  datasource:
    db:
      type: com.alibaba.druid.pool.DruidDataSource
      driverClassName: com.mysql.jdbc.Driver
      minIdle: 50
      initialSize: 50
      maxActive: 300
      maxWait: 1000
      testOnBorrow: false
      testOnReturn: true
      testWhileIdle: true
      validationQuery: select 1
      validationQueryTimeout: 1
      timeBetweenEvictionRunsMillis: 5000
      ConnectionErrorRetryAttempts: 3
      NotFullTimeoutRetryCount: 3
      numTestsPerEvictionRun: 10
      minEvictableIdleTimeMillis: 480000
      maxEvictableIdleTimeMillis: 480000
      keepAliveBetweenTimeMillis: 480000
      keepalive: true
      poolPreparedStatements: true
      maxPoolPreparedStatementPerConnectionSize: 512
      maxOpenPreparedStatements: 512
    master: #master db
      type: com.alibaba.druid.pool.DruidDataSource
      driverClassName: com.mysql.jdbc.Driver
      url: jdbc:mysql://localhost:3306/ecom?useUnicode=true&characterEncoding=utf-8&useSSL=false&useAffectedRows=true&autoReconnect=true
      username: root
      password: 111111
    slaver: #slaver db
      type: com.alibaba.druid.pool.DruidDataSource
      driverClassName: com.mysql.jdbc.Driver
      url: jdbc:mysql://localhost:3307/ecom?useUnicode=true&characterEncoding=utf-8&useSSL=false&useAffectedRows=true&autoReconnect=true
      username: root
      password: 111111     
server:
  port: 9080
  tomcat:
    max-http-post-size: -1
  max-http-header-size: 10240000

spring:
  application:
    name: skypayment
  servlet:
    multipart:
      max-file-size: 10MB
      max-request-size: 10MB
    context-path: /skypayment

  #Configure rabbitMq server
  rabbitmq:
    addresses: localhost:5672
    username: admin
    password: admin
    #The virtual host can not be set. Use the server default host
    virtual-host: /
    publisher-confirm-type: CORRELATED
    listener:
       ## simple type
       simple:
         #Minimum number of consumers
         concurrency: 32
         #Largest number of consumers
         maxConcurrency: 64
         #Specify how many messages a request can process. If there are transactions, the number must be greater than or equal to the number of transactions
         prefetch: 32
         retry: 
           enabled: false
#rabbitmq timeout is used for queue timeout
queue:
  expire: 1000

RabbitMqConfig.java

/**
 * System project name: org mk.demo. rabbitmqdemo. config RabbitMqConfig. java
 * 
 * Nov 19, 2021-11:30:43 AM 2021XX Company - All Rights Reserved
 * 
 */
package org.mk.demo.skypayment.config.mq;

import org.springframework.amqp.core.Binding;
import org.springframework.amqp.core.BindingBuilder;
import org.springframework.amqp.core.ExchangeBuilder;
import org.springframework.amqp.core.FanoutExchange;
import org.springframework.amqp.core.Queue;
import org.springframework.amqp.core.QueueBuilder;
import org.springframework.amqp.core.TopicExchange;
import org.springframework.amqp.rabbit.connection.ConnectionFactory;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.amqp.support.converter.Jackson2JsonMessageConverter;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.stereotype.Component;

/**
 * 
 * RabbitMqConfig
 * 
 * 
 * Nov 19, 2021 11:30:43 AM
 * 
 * @version 1.0.0
 * 
 */
@Component
public class RabbitMqConfig {
    /**
     * It mainly tests a dead letter queue, which mainly realizes delayed consumption. The principle is to send the message to the normal queue first. The normal queue has a timeout time. When the time is reached, it will be automatically sent to the dead letter queue, and then the consumer will consume the messages in the dead letter queue
     */
    public static final String PAYMENT_EXCHANGE = "payment.exchange";
    public static final String PAYMENT_DL_EXCHANGE = "payment.dl.exchange";
    public static final String PAYMENT_QUEUE = "payment.queue";
    public static final String PAYMENT_DEAD_QUEUE = "payment.queue.dead";

    public static final String PAYMENT_FANOUT_EXCHANGE = "paymentFanoutExchange";
    /**
     * The unit is microseconds
     */
    @Value("${queue.expire:5000}")
    private long queueExpire;

    /**
     * Create a general switch
     */
    @Bean
    public TopicExchange paymentExchange() {
        return (TopicExchange)ExchangeBuilder.topicExchange(PAYMENT_EXCHANGE).durable(true).build();
    }

    /**
     * Create a dead letter switch
     */
    @Bean
    public TopicExchange paymentExchangeDl() {
        return (TopicExchange)ExchangeBuilder.topicExchange(PAYMENT_DL_EXCHANGE).durable(true).build();
    }

    /**
     * Create a normal queue
     */
    @Bean
    public Queue paymentQueue() {
        return QueueBuilder.durable(PAYMENT_QUEUE).withArgument("x-dead-letter-exchange", PAYMENT_DL_EXCHANGE)// Set up dead letter switch
            .withArgument("x-message-ttl", queueExpire).withArgument("x-dead-letter-routing-key", PAYMENT_DEAD_QUEUE)// Set dead letter routingKey
            .build();
    }

    /**
     * Create a dead letter queue
     */
    @Bean
    public Queue paymentDelayQueue() {
        return QueueBuilder.durable(PAYMENT_DEAD_QUEUE).build();
    }

    /**
     * Bind dead letter queue
     */
    @Bean
    public Binding bindDeadBuilders() {
        return BindingBuilder.bind(paymentDelayQueue()).to(paymentExchangeDl()).with(PAYMENT_DEAD_QUEUE);
    }

    /**
     * Bind normal queue
     *
     * @return
     */
    @Bean
    public Binding bindBuilders() {
        return BindingBuilder.bind(paymentQueue()).to(paymentExchange()).with(PAYMENT_QUEUE);
    }

    /**
     * Broadcast switch
     *
     * @return
     */
    @Bean
    public FanoutExchange fanoutExchange() {
        return new FanoutExchange(PAYMENT_FANOUT_EXCHANGE);
    }

    @Bean
    public RabbitTemplate rabbitTemplate(final ConnectionFactory connectionFactory) {
        final RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
        rabbitTemplate.setMessageConverter(producerJackson2MessageConverter());
        return rabbitTemplate;
    }

    @Bean
    public Jackson2JsonMessageConverter producerJackson2MessageConverter() {
        return new Jackson2JsonMessageConverter();
    }
}

RabbitMqListenerConfig.java

/**
 * System project name: org mk.demo. skypayment. config. mq RabbitMqListenerConfig. java
 * 
 * Feb 3, 2022-12:35:21 AM 2022XX Company - All Rights Reserved
 * 
 */
package org.mk.demo.skypayment.config.mq;

import org.springframework.amqp.rabbit.annotation.RabbitListenerConfigurer;
import org.springframework.amqp.rabbit.listener.RabbitListenerEndpointRegistrar;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.messaging.converter.MappingJackson2MessageConverter;
import org.springframework.messaging.handler.annotation.support.DefaultMessageHandlerMethodFactory;
import org.springframework.messaging.handler.annotation.support.MessageHandlerMethodFactory;

/**
 * 
 * RabbitMqListenerConfig
 * 
 * 
 * Feb 3, 2022 12:35:21 AM
 * 
 * @version 1.0.0
 * 
 */
@Configuration
public class RabbitMqListenerConfig implements RabbitListenerConfigurer {

    /* (non-Javadoc)
     * @see org.springframework.amqp.rabbit.annotation.RabbitListenerConfigurer#configureRabbitListeners(org.springframework.amqp.rabbit.listener.RabbitListenerEndpointRegistrar)
     */
    @Override
    public void configureRabbitListeners(RabbitListenerEndpointRegistrar registor) {
        registor.setMessageHandlerMethodFactory(messageHandlerMethodFactory());

    }

    @Bean
    MessageHandlerMethodFactory messageHandlerMethodFactory() {
        DefaultMessageHandlerMethodFactory messageHandlerMethodFactory = new DefaultMessageHandlerMethodFactory();
        messageHandlerMethodFactory.setMessageConverter(consumerJackson2MessageConverter());
        return messageHandlerMethodFactory;
    }

    @Bean
    public MappingJackson2MessageConverter consumerJackson2MessageConverter() {
        return new MappingJackson2MessageConverter();
    }
}

When the spring boot project starts, it will establish the following main contents in RabbitMQ

Two queues, one normal queue and one with Queue ended by dead;

Two exchanges, one normal exchange corresponding to the normal queue and the other dl.exchange corresponds to dead end queue.

Note:

When running my project example, you should remember to assign permissions to the connected users of rabbitmq on your own machine to establish queue and exchange. How to assign permissions to rabbitmq's own users is too much on the Internet to be explained in detail here.

Payment in the above The queue has an expiration time. Here we set it to 1 second to simulate the timeout of normal payment callback. Once the queue reaches the 1-second timeout, it will be forwarded to payment dl. In exchange, and then at this time, if your application corresponds to payment.queue.dead in payment.dl.exchange through "listening", you can get this message. After getting this message, you will get the "business code processing" of your normal "business timeout compensation". Here we call the update by id method. The following code block.

PaymentDao.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd">
<mapper namespace="org.mk.demo.skypayment.dao.PaymentDao">
	<resultMap id="paymentResult"
		type="org.mk.demo.skypayment.vo.PaymentBean">
		<id column="pay_id" property="payId" javaType="Integer" />
		<result column="status" property="status" javaType="Integer" />
		<result column="transf_amount" property="transfAmount" javaType="Double" />
	</resultMap>

	
	<update id="updatePaymentStatusByLimit">
		update sky_payment set status='2' where status='1' limit 3000
	</update>
	
	<update id="updatePaymentStatusById" parameterType="org.mk.demo.skypayment.vo.PaymentBean">
		update sky_payment set status=#{status}  where pay_id=#{payId}
	</update>

</mapper>

Corresponding paymentdao java

/**
 * System project name: org mk.demo. skypayment. dao PaymentDao. java
 * 
 * Feb 1, 2022-11:13:38 PM 2022XX Company - All Rights Reserved
 * 
 */
package org.mk.demo.skypayment.dao;

import org.springframework.stereotype.Repository;

import java.util.List;

import org.mk.demo.skypayment.vo.PaymentBean;

/**
 * 
 * PaymentDao
 * 
 * 
 * Feb 1, 2022 11:13:38 PM
 * 
 * @version 1.0.0
 * 
 */
@Repository
public interface PaymentDao {

    public int updatePaymentStatusByLimit();

    public int updatePaymentStatusById(PaymentBean payment);

}

Publisher.java

It is used to generate requests for normal payment requests to third-party payment channels or payment gateways. This class is used to set a timeout for this queue while submitting payment requests normally.

If there is a response within the timeout, the queue will not generate a "corresponding dead letter queue";
The reverse is "if you receive the corresponding order flow in the dead letter queue, it must mean that this payment request has not been responded within the timeout allowed by the business, so it needs to be compensated";

In order to simulate this scenario, we will not process this request, so this request will always be in the dead letter queue

/**
 * System project name: org mk.demo. rabbitmqdemo. service Publisher. java
 * 
 * Nov 19, 2021-11:38:39 AM 2021XX Company - All Rights Reserved
 * 
 */
package org.mk.demo.skypayment.service;

import org.mk.demo.skypayment.config.mq.RabbitMqConfig;
import org.mk.demo.skypayment.vo.PaymentBean;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.amqp.support.converter.Jackson2JsonMessageConverter;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

/**
 * 
 * Publisher
 * 
 * 
 * Nov 19, 2021 11:38:39 AM
 * 
 * @version 1.0.0
 * 
 */
@Component

public class Publisher {
    private Logger logger = LoggerFactory.getLogger(this.getClass());
    @Autowired
    private RabbitTemplate rabbitTemplate;

    public void publishPaymentStatusChange(PaymentBean payment) {
        try {
            rabbitTemplate.setMessageConverter(new Jackson2JsonMessageConverter());
            rabbitTemplate.convertAndSend(RabbitMqConfig.PAYMENT_EXCHANGE, RabbitMqConfig.PAYMENT_QUEUE, payment);
        } catch (Exception ex) {
            logger.error(">>>>>>publish exception: " + ex.getMessage(), ex);
        }
    }
}

Subscriber.java

It is used to realize the business compensation operation after listening to the dead letter queue and getting the message.

/**
 * System project name: org mk.demo. rabbitmqdemo. service Subscriber. java
 * 
 * Nov 19, 2021-11:47:02 AM 2021XX Company - All Rights Reserved
 * 
 */
package org.mk.demo.skypayment.service;

import javax.annotation.Resource;

import org.mk.demo.skypayment.config.mq.RabbitMqConfig;
import org.mk.demo.skypayment.vo.PaymentBean;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.amqp.rabbit.annotation.RabbitListener;
import org.springframework.stereotype.Component;

/**
 * 
 * Subscriber
 * 
 * 
 * Nov 19, 2021 11:47:02 AM
 * 
 * @version 1.0.0
 * 
 */
@Component
public class Subscriber {
    private Logger logger = LoggerFactory.getLogger(this.getClass());

    @Resource
    private PaymentService paymentService;

    @RabbitListener(queues = RabbitMqConfig.PAYMENT_DEAD_QUEUE)
    public void receiveDL(PaymentBean payment) {
        try {
            if (payment != null) {
                logger.info(">>>>>>Get data from dead letter queue and develop changes: payId->{} status->{} transfAmount->{}", payment.getPayId(),
                    payment.getStatus(), payment.getTransfAmount());
                int records = paymentService.updatePaymentStatusById(payment);
                logger.info(">>>>>>modify: {}", records);
            }
        } catch (Exception ex) {
            logger.error(">>>>>>Subscriber from dead queue exception: " + ex.getMessage(), ex);
        }
    }
}

Run the example of using dead letter queue to realize business compensation

Next, let's run with two scripts. So many requests are pressed into the front end, and each request will generate a dead letter.

To this end, we first make the two spring boot templates into a cluster mode, as shown in the following nginix settings:

The core configuration of spring boot cluster as agent in nginix

    upstream skypayment-lb {
      server   localhost:9080 weight=1 fail_timeout=1s;
      server   localhost:9081 weight=1 fail_timeout=1s;
    }
    

    server {

        location /skypayment/ {
                port_in_redirect on;
                # Load configuration
                proxy_pass http://skypayment-lb/;
                proxy_redirect  off;
                proxy_set_header Host $host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                add_header backendIP $upstream_addr;
                add_header backendCode $upstream_status;
        }

It has been running for nearly 3 minutes without any deadlock. Because it is impossible to generate deadlock, it is because every update in the database is by id.

Let's continue to look at rabbitmq's console http://localhost:15672 The status of the squadron.

It is found that there is a backlog of 700-1000 unack queues. This is not a problem, because the front-end requests are too large. You can think that more than 23000 payment requests need to be processed concurrently!

If your enterprise has so many payments a day, Congratulations, you must earn more than 300 million a year.

In order to solve this continuous backlog of unack messages, I... Start one more template to make the application run from the original two templates to three templates at the same time.

After less than 10 seconds, the unack queue of the whole RabbitMQ city was instantly reduced to single digits.

This is mentioned in cloud native. Your application cannot be limited by the corresponding resource constraints, and the computing power of the system will expand with the horizontal elastic expansion of the application.

Finally, after more than 23000 requests at the current end, RabbitMQ continued to change the payment status in some unprocessed queues for about 10 seconds, and the whole system was calm.

0 error, 0 deadlock, high system response, large amount of spitting and swallowing, micro service, cloud native.

Attachment

parent-pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>org.mk.demo</groupId>
	<artifactId>springboot-demo</artifactId>
	<version>0.0.1</version>
	<packaging>pom</packaging>
	<properties>
		<java.version>1.8</java.version>
		<jacoco.version>0.8.3</jacoco.version>
		<aldi-sharding.version>0.0.1</aldi-sharding.version>
		<spring-boot.version>2.4.2</spring-boot.version>
		<!-- <spring-boot.version>2.3.1.RELEASE</spring-boot.version> -->
		<!-- spring-boot.version>2.0.6.RELEASE</spring-boot.version> <spring-cloud-zk-discovery.version>2.1.3.RELEASE</spring-cloud-zk-discovery.version -->
		<zookeeper.version>3.4.13</zookeeper.version>
		<!-- <spring-cloud.version>Greenwich.SR5</spring-cloud.version> -->
		<spring-cloud.version>2020.0.1</spring-cloud.version>
		<dubbo.version>2.7.3</dubbo.version>
		<curator-framework.version>4.0.1</curator-framework.version>
		<curator-recipes.version>2.8.0</curator-recipes.version>
		<!-- druid.version>1.1.20</druid.version -->
		<druid.version>1.2.6</druid.version>
		<guava.version>27.0.1-jre</guava.version>
		<fastjson.version>1.2.59</fastjson.version>
		<dubbo-registry-nacos.version>2.7.3</dubbo-registry-nacos.version>
		<nacos-client.version>1.1.4</nacos-client.version>
		<!-- mysql-connector-java.version>8.0.13</mysql-connector-java.version -->
		<mysql-connector-java.version>5.1.46</mysql-connector-java.version>
		<disruptor.version>3.4.2</disruptor.version>
		<aspectj.version>1.8.13</aspectj.version>
		<spring.data.redis>1.8.14-RELEASE</spring.data.redis>
		<seata.version>1.0.0</seata.version>
		<netty.version>4.1.42.Final</netty.version>
		<nacos.spring.version>0.1.4</nacos.spring.version>
		<lombok.version>1.16.22</lombok.version>
		<javax.servlet.version>3.1.0</javax.servlet.version>
		<mybatis.version>2.1.0</mybatis.version>
		<pagehelper-mybatis.version>1.2.3</pagehelper-mybatis.version>
		<spring.kafka.version>1.3.10.RELEASE</spring.kafka.version>
		<kafka.client.version>1.0.2</kafka.client.version>
		<shardingsphere.jdbc.version>4.0.0</shardingsphere.jdbc.version>
		<xmemcached.version>2.4.6</xmemcached.version>
		<swagger.version>2.9.2</swagger.version>
		<swagger.bootstrap.ui.version>1.9.6</swagger.bootstrap.ui.version>
		<swagger.model.version>1.5.23</swagger.model.version>
		<swagger-annotations.version>1.5.22</swagger-annotations.version>
		<swagger-models.version>1.5.22</swagger-models.version>
		<swagger-bootstrap-ui.version>1.9.5</swagger-bootstrap-ui.version>
		<sky-sharding-jdbc.version>0.0.1</sky-sharding-jdbc.version>
		<cxf.version>3.1.6</cxf.version>
		<jackson-databind.version>2.11.1</jackson-databind.version>
		<gson.version>2.8.6</gson.version>
		<groovy.version>2.5.8</groovy.version>
		<logback-ext-spring.version>0.1.4</logback-ext-spring.version>
		<jcl-over-slf4j.version>1.7.25</jcl-over-slf4j.version>
		<spock-spring.version>2.0-M2-groovy-2.5</spock-spring.version>
		<xxljob.version>2.2.0</xxljob.version>
		<java-jwt.version>3.10.0</java-jwt.version>
		<commons-lang.version>2.6</commons-lang.version>
		<hutool-crypto.version>5.0.0</hutool-crypto.version>
		<maven.compiler.source>${java.version}</maven.compiler.source>
		<maven.compiler.target>${java.version}</maven.compiler.target>
		<compiler.plugin.version>3.8.1</compiler.plugin.version>
		<war.plugin.version>3.2.3</war.plugin.version>
		<jar.plugin.version>3.1.1</jar.plugin.version>
		<quartz.version>2.2.3</quartz.version>
		<h2.version>1.4.197</h2.version>
		<zkclient.version>3.4.14</zkclient.version>
		<httpcore.version>4.4.10</httpcore.version>
		<httpclient.version>4.5.6</httpclient.version>
		<mockito-core.version>3.0.0</mockito-core.version>
		<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
		<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
		<oseq-aldi.version>2.0.22-RELEASE</oseq-aldi.version>
		<poi.version>4.1.0</poi.version>
		<poi-ooxml.version>4.1.0</poi-ooxml.version>
		<poi-ooxml-schemas.version>4.1.0</poi-ooxml-schemas.version>
		<dom4j.version>1.6.1</dom4j.version>
		<xmlbeans.version>3.1.0</xmlbeans.version>
		<java-jwt.version>3.10.0</java-jwt.version>
		<commons-lang.version>2.6</commons-lang.version>
		<hutool-crypto.version>5.0.0</hutool-crypto.version>
		<nacos-discovery.version>2.2.5.RELEASE</nacos-discovery.version>
		<spring-cloud-alibaba.version>2.2.1.RELEASE</spring-cloud-alibaba.version>
		<redission.version>3.16.1</redission.version>
		<log4j2.version>2.17.1</log4j2.version>
		<redis-common.version>0.0.1</redis-common.version>
		<common-util.version>0.0.1</common-util.version>
		<db-common.version>0.0.1</db-common.version>
	</properties>
	<dependencyManagement>
		<dependencies>
			<!-- redis -->
			<dependency>
				<groupId>org.redisson</groupId>
				<artifactId>redisson-spring-boot-starter</artifactId>
				<version>${redission.version}</version>
				<!-- <exclusions> <exclusion> <groupId>org.redisson</groupId> <artifactId>redisson-spring-data-23</artifactId> 
					</exclusion> </exclusions> -->
			</dependency>
			<dependency>
				<groupId>org.redisson</groupId>
				<artifactId>redisson-spring-data-21</artifactId>
				<version>${redission.version}</version>
			</dependency>
			<dependency>
				<groupId>com.alibaba.cloud</groupId>
				<artifactId>spring-cloud-alibaba-dependencies</artifactId>
				<version>${spring-cloud-alibaba.version}</version>
				<type>pom</type>
				<scope>import</scope>
			</dependency>
			<dependency>
				<groupId>com.alibaba.cloud</groupId>
				<artifactId>spring-cloud-starter-alibaba-nacos-discovery
				</artifactId>
				<version>${nacos-discovery.version}</version>
			</dependency>
			<dependency>
				<groupId>com.aldi.jdbc</groupId>
				<artifactId>sharding</artifactId>
				<version>${aldi-sharding.version}</version>
			</dependency>
			<!-- jwt -->
			<dependency>
				<groupId>com.auth0</groupId>
				<artifactId>java-jwt</artifactId>
				<version>${java-jwt.version}</version>
			</dependency>
			<dependency>
				<groupId>cn.hutool</groupId>
				<artifactId>hutool-crypto</artifactId>
				<version>${hutool-crypto.version}</version>
			</dependency>
			<!-- poi must start -->
			<dependency>
				<groupId>org.apache.poi</groupId>
				<artifactId>poi</artifactId>
				<version>${poi.version}</version>
			</dependency>
			<dependency>
				<groupId>org.apache.poi</groupId>
				<artifactId>poi-ooxml</artifactId>
				<version>${poi-ooxml.version}</version>
			</dependency>
			<dependency>
				<groupId>org.apache.poi</groupId>
				<artifactId>poi-ooxml</artifactId>
				<version>${poi-ooxml.version}</version>
			</dependency>
			<dependency>
				<groupId>org.apache.poi</groupId>
				<artifactId>poi-ooxml-schemas</artifactId>
				<version>${poi-ooxml-schemas.version}</version>
			</dependency>
			<dependency>
				<groupId>dom4j</groupId>
				<artifactId>dom4j</artifactId>
				<version>${dom4j.version}</version>
			</dependency>
			<dependency>
				<groupId>org.apache.xmlbeans</groupId>
				<artifactId>xmlbeans</artifactId>
				<version>${xmlbeans.version}</version>
			</dependency>
			<!-- poi must end -->
			<dependency>
				<groupId>com.odianyun.architecture</groupId>
				<artifactId>oseq-aldi</artifactId>
				<version>${oseq-aldi.version}</version>
			</dependency>
			<dependency>
				<groupId>org.apache.httpcomponents</groupId>
				<artifactId>httpcore</artifactId>
				<version>${httpcore.version}</version>
			</dependency>

			<dependency>
				<groupId>org.apache.httpcomponents</groupId>
				<artifactId>httpclient</artifactId>
				<version>${httpclient.version}</version>
			</dependency>
			<dependency>
				<groupId>org.apache.zookeeper</groupId>
				<artifactId>zookeeper</artifactId>
				<version>${zkclient.version}</version>
				<exclusions>
					<exclusion>
						<artifactId>log4j</artifactId>
						<groupId>log4j</groupId>
					</exclusion>
					<exclusion>
						<groupId>org.slf4j</groupId>
						<artifactId>slf4j-log4j12</artifactId>
					</exclusion>
				</exclusions>
			</dependency>
			<!--quartz rely on -->
			<dependency>
				<groupId>org.quartz-scheduler</groupId>
				<artifactId>quartz</artifactId>
				<version>${quartz.version}</version>
			</dependency>
			<dependency>
				<groupId>org.quartz-scheduler</groupId>
				<artifactId>quartz-jobs</artifactId>
				<version>${quartz.version}</version>
			</dependency>
			<!-- spring cloud base -->
			<dependency>
				<groupId>org.springframework.cloud</groupId>
				<artifactId>spring-cloud-dependencies</artifactId>
				<version>Hoxton.SR7</version>
				<type>pom</type>
				<scope>import</scope>
			</dependency>

			<!-- use mockito -->
			<dependency>
				<groupId>org.mockito</groupId>
				<artifactId>mockito-core</artifactId>
				<version>${mockito-core.version}</version>
				<scope>test</scope>
			</dependency>
			<!-- jwt token -->
			<dependency>
				<groupId>com.auth0</groupId>
				<artifactId>java-jwt</artifactId>
				<version>${java-jwt.version}</version>
			</dependency>
			<dependency>
				<groupId>cn.hutool</groupId>
				<artifactId>hutool-crypto</artifactId>
				<version>${hutool-crypto.version}</version>
			</dependency>

			<dependency>
				<groupId>org.springframework.boot</groupId>
				<artifactId>spring-boot-starter-actuator</artifactId>
				<version>${spring-boot.version}</version>
			</dependency>
			<!-- Logback -->
			<dependency>
				<groupId>org.logback-extensions</groupId>
				<artifactId>logback-ext-spring</artifactId>
				<version>${logback-ext-spring.version}</version>
			</dependency>
			<dependency>
				<groupId>org.slf4j</groupId>
				<artifactId>jcl-over-slf4j</artifactId>
				<version>${jcl-over-slf4j.version}</version>
			</dependency>

			<!-- Database for unit testing -->
			<dependency>
				<groupId>com.h2database</groupId>
				<artifactId>h2</artifactId>
				<version>${h2.version}</version>
			</dependency>

			<dependency>
				<groupId>org.apache.zookeeper</groupId>
				<artifactId>zookeeper</artifactId>
				<version>${zookeeper.version}</version>
				<exclusions>
					<exclusion>
						<groupId>org.slf4j</groupId>
						<artifactId>slf4j-log4j12</artifactId>
					</exclusion>
					<exclusion>
						<groupId>log4j</groupId>
						<artifactId>log4j</artifactId>
					</exclusion>
				</exclusions>
			</dependency>

			<!-- xxl-rpc-core -->
			<dependency>
				<groupId>com.xuxueli</groupId>
				<artifactId>xxl-job-core</artifactId>
				<version>${xxljob.version}</version>
			</dependency>
			<dependency>
				<groupId>org.springframework.boot</groupId>
				<artifactId>spring-boot-starter-test</artifactId>
				<version>${spring-boot.version}</version>
				<scope>test</scope>
				<exclusions>
					<exclusion>
						<groupId>org.springframework.boot</groupId>
						<artifactId>spring-boot-starter-logging</artifactId>
					</exclusion>
					<exclusion>
						<groupId>org.slf4j</groupId>
						<artifactId>slf4j-log4j12</artifactId>
					</exclusion>
				</exclusions>
			</dependency>
			<dependency>
				<groupId>org.spockframework</groupId>
				<artifactId>spock-core</artifactId>
				<version>1.3-groovy-2.4</version>
				<scope>test</scope>
			</dependency>
			<dependency>
				<groupId>org.spockframework</groupId>
				<artifactId>spock-spring</artifactId>
				<version>1.3-RC1-groovy-2.4</version>
				<scope>test</scope>
			</dependency>
			<dependency>
				<groupId>org.codehaus.groovy</groupId>
				<artifactId>groovy-all</artifactId>
				<version>2.4.6</version>
			</dependency>

			<dependency>
				<groupId>com.google.code.gson</groupId>
				<artifactId>gson</artifactId>
				<version>${gson.version}</version>
			</dependency>
			<dependency>
				<groupId>com.fasterxml.jackson.core</groupId>
				<artifactId>jackson-databind</artifactId>
				<version>${jackson-databind.version}</version>
			</dependency>
			<dependency>
				<groupId>org.springframework.boot</groupId>
				<artifactId>spring-boot-starter-web-services</artifactId>
				<version>${spring-boot.version}</version>
			</dependency>
			<dependency>
				<groupId>org.apache.cxf</groupId>
				<artifactId>cxf-rt-frontend-jaxws</artifactId>
				<version>${cxf.version}</version>
			</dependency>
			<dependency>
				<groupId>org.apache.cxf</groupId>
				<artifactId>cxf-rt-transports-http</artifactId>
				<version>${cxf.version}</version>
			</dependency>
			<dependency>
				<groupId>org.springframework.boot</groupId>
				<artifactId>spring-boot-starter-security</artifactId>
				<version>${spring-boot.version}</version>
			</dependency>

			<dependency>
				<groupId>io.github.swagger2markup</groupId>
				<artifactId>swagger2markup</artifactId>
				<version>1.3.1</version>
			</dependency>
			<dependency>
				<groupId>io.springfox</groupId>
				<artifactId>springfox-swagger2</artifactId>
				<version>${swagger.version}</version>
			</dependency>

			<dependency>
				<groupId>io.springfox</groupId>
				<artifactId>springfox-swagger-ui</artifactId>
				<version>${swagger.version}</version>
			</dependency>

			<dependency>
				<groupId>com.github.xiaoymin</groupId>
				<artifactId>swagger-bootstrap-ui</artifactId>
				<version>${swagger-bootstrap-ui.version}</version>
			</dependency>

			<dependency>
				<groupId>io.swagger</groupId>
				<artifactId>swagger-annotations</artifactId>
				<version>${swagger-annotations.version}</version>
			</dependency>

			<dependency>
				<groupId>io.swagger</groupId>
				<artifactId>swagger-models</artifactId>
				<version>${swagger-models.version}</version>
			</dependency>
			<dependency>
				<groupId>org.sky</groupId>
				<artifactId>sky-sharding-jdbc</artifactId>
				<version>${sky-sharding-jdbc.version}</version>
			</dependency>
			<dependency>
				<groupId>com.googlecode.xmemcached</groupId>
				<artifactId>xmemcached</artifactId>
				<version>${xmemcached.version}</version>
			</dependency>
			<dependency>
				<groupId>org.apache.shardingsphere</groupId>
				<artifactId>sharding-jdbc-core</artifactId>
				<version>${shardingsphere.jdbc.version}</version>
			</dependency>
			<!-- <dependency> <groupId>org.springframework.kafka</groupId> <artifactId>spring-kafka</artifactId> 
				<version>${spring.kafka.version}</version> </dependency> <dependency> <groupId>org.apache.kafka</groupId> 
				<artifactId>kafka-clients</artifactId> <version>${kafka.client.version}</version> 
				</dependency> -->
			<dependency>
				<groupId>org.springframework.kafka</groupId>
				<artifactId>spring-kafka</artifactId>
				<version>1.3.10.RELEASE</version>
			</dependency>
			<dependency>
				<groupId>org.mybatis.spring.boot</groupId>
				<artifactId>mybatis-spring-boot-starter</artifactId>
				<version>${mybatis.version}</version>
			</dependency>
			<dependency>
				<groupId>com.github.pagehelper</groupId>
				<artifactId>pagehelper-spring-boot-starter</artifactId>
				<version>${pagehelper-mybatis.version}</version>
			</dependency>
			<dependency>
				<groupId>org.springframework.boot</groupId>
				<artifactId>spring-boot-starter-web</artifactId>
				<version>${spring-boot.version}</version>
				<exclusions>
					<exclusion>
						<groupId>org.slf4j</groupId>
						<artifactId>slf4j-log4j12</artifactId>
					</exclusion>
					<exclusion>
						<groupId>org.springframework.boot</groupId>
						<artifactId>spring-boot-starter-logging</artifactId>
					</exclusion>
				</exclusions>
			</dependency>
			<dependency>
				<groupId>org.springframework.boot</groupId>
				<artifactId>spring-boot-dependencies</artifactId>
				<version>${spring-boot.version}</version>
				<type>pom</type>
				<scope>import</scope>
			</dependency>
			<dependency>
				<groupId>org.apache.dubbo</groupId>
				<artifactId>dubbo-spring-boot-starter</artifactId>
				<version>${dubbo.version}</version>
				<exclusions>
					<exclusion>
						<groupId>org.slf4j</groupId>
						<artifactId>slf4j-log4j12</artifactId>
					</exclusion>
					<exclusion>
						<groupId>org.springframework.boot</groupId>
						<artifactId>spring-boot-starter-logging</artifactId>
					</exclusion>
				</exclusions>
			</dependency>
			<dependency>
				<groupId>org.apache.dubbo</groupId>
				<artifactId>dubbo</artifactId>
				<version>${dubbo.version}</version>
				<exclusions>
					<exclusion>
						<groupId>javax.servlet</groupId>
						<artifactId>servlet-api</artifactId>
					</exclusion>
				</exclusions>
			</dependency>
			<dependency>
				<groupId>org.apache.curator</groupId>
				<artifactId>curator-framework</artifactId>
				<version>${curator-framework.version}</version>
			</dependency>

			<dependency>
				<groupId>org.apache.curator</groupId>
				<artifactId>curator-recipes</artifactId>
				<version>${curator-recipes.version}</version>
			</dependency>
			<dependency>
				<groupId>mysql</groupId>
				<artifactId>mysql-connector-java</artifactId>
				<version>${mysql-connector-java.version}</version>
			</dependency>
			<dependency>
				<groupId>com.alibaba</groupId>
				<artifactId>druid</artifactId>
				<version>${druid.version}</version>
			</dependency>
			<dependency>
				<groupId>com.alibaba</groupId>
				<artifactId>druid-spring-boot-starter</artifactId>
				<version>${druid.version}</version>
			</dependency>
			<dependency>
				<groupId>com.lmax</groupId>
				<artifactId>disruptor</artifactId>
				<version>${disruptor.version}</version>
			</dependency>
			<dependency>
				<groupId>com.google.guava</groupId>
				<artifactId>guava</artifactId>
				<version>${guava.version}</version>
			</dependency>
			<dependency>
				<groupId>com.alibaba</groupId>
				<artifactId>fastjson</artifactId>
				<version>${fastjson.version}</version>
			</dependency>
			<dependency>
				<groupId>org.apache.dubbo</groupId>
				<artifactId>dubbo-registry-nacos</artifactId>
				<version>${dubbo-registry-nacos.version}</version>
			</dependency>
			<dependency>
				<groupId>com.alibaba.nacos</groupId>
				<artifactId>nacos-client</artifactId>
				<version>${nacos-client.version}</version>
			</dependency>
			<dependency>
				<groupId>org.aspectj</groupId>
				<artifactId>aspectjweaver</artifactId>
				<version>${aspectj.version}</version>
			</dependency>
			<dependency>
				<groupId>org.springframework.boot</groupId>
				<artifactId>spring-boot-starter-data-redis</artifactId>
				<version>${spring-boot.version}</version>
			</dependency>
			<dependency>
				<groupId>io.seata</groupId>
				<artifactId>seata-all</artifactId>
				<version>${seata.version}</version>
			</dependency>
			<dependency>
				<groupId>io.netty</groupId>
				<artifactId>netty-all</artifactId>
				<version>${netty.version}</version>
			</dependency>
			<dependency>
				<groupId>org.projectlombok</groupId>
				<artifactId>lombok</artifactId>
				<version>${lombok.version}</version>
			</dependency>
			<!-- https://mvnrepository.com/artifact/com.alibaba.boot/nacos-config-spring-boot-starter -->
			<dependency>
				<groupId>com.alibaba.boot</groupId>
				<artifactId>nacos-config-spring-boot-starter</artifactId>
				<version>${nacos.spring.version}</version>
				<exclusions>
					<exclusion>
						<artifactId>nacos-client</artifactId>
						<groupId>com.alibaba.nacos</groupId>
					</exclusion>
				</exclusions>
			</dependency>

			<dependency>
				<groupId>net.sourceforge.groboutils</groupId>
				<artifactId>groboutils-core</artifactId>
				<version>5</version>
			</dependency>
			<dependency>
				<groupId>commons-lang</groupId>
				<artifactId>commons-lang</artifactId>
				<version>${commons-lang.version}</version>
			</dependency>
		</dependencies>
	</dependencyManagement>
	<modules>
		<module>rabbitmq-demo</module>
		<module>redis-demo</module>
		<module>db-demo</module>
		<module>threadpool-demo</module>
		<module>oseq-demo</module>
		<module>osoa-demo</module>
		<module>ody-channel-web</module>
		<module>export-to-excel-demo</module>
		<module>ody-smartds</module>
		<module>import-to-db</module>
		<module>mvc1</module>
		<module>mvc2</module>
		<module>datastructure-demo</module>
		<module>sharding-jdbc-demo</module>
		<module>sm2-demo</module>
		<module>redis-common</module>
		<module>redis-doublebuffer-demo</module>
		<module>common-util</module>
		<module>db-common</module>
		<module>skypayment</module>
	</modules>
</project>

pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<parent>
		<groupId>org.mk.demo</groupId>
		<artifactId>springboot-demo</artifactId>
		<version>0.0.1</version>
	</parent>
	<artifactId>skypayment</artifactId>
	<dependencies>
		<!-- mq must -->
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-amqp</artifactId>
			<exclusions>
				<exclusion>
					<groupId>org.springframework.boot</groupId>
					<artifactId>spring-boot-starter-logging</artifactId>
				</exclusion>
			</exclusions>
		</dependency>
		<!-- mysql must -->
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-jdbc</artifactId>
			<exclusions>
				<exclusion>
					<groupId>org.springframework.boot</groupId>
					<artifactId>spring-boot-starter-logging</artifactId>
				</exclusion>
			</exclusions>
		</dependency>
		<dependency>
			<groupId>mysql</groupId>
			<artifactId>mysql-connector-java</artifactId>
		</dependency>
		<dependency>
			<groupId>com.alibaba</groupId>
			<artifactId>druid</artifactId>
		</dependency>
		<!-- redis must -->
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-data-redis</artifactId>
			<exclusions>
				<exclusion>
					<groupId>org.springframework.boot</groupId>
					<artifactId>spring-boot-starter-logging</artifactId>
				</exclusion>
				<exclusion>
					<groupId>org.slf4j</groupId>
					<artifactId>slf4j-log4j12</artifactId>
				</exclusion>
			</exclusions>
		</dependency>
		<dependency>
			<groupId>org.mybatis.spring.boot</groupId>
			<artifactId>mybatis-spring-boot-starter</artifactId>
		</dependency>
		<!-- jedis must -->
		<dependency>
			<groupId>redis.clients</groupId>
			<artifactId>jedis</artifactId>
		</dependency>
		<!--redisson must start -->
		<dependency>
			<groupId>org.redisson</groupId>
			<artifactId>redisson-spring-boot-starter</artifactId>
			<version>3.13.6</version>
			<exclusions>
				<exclusion>
					<groupId>org.redisson</groupId>
					<artifactId>redisson-spring-data-23</artifactId>
				</exclusion>
			</exclusions>
		</dependency>
		<dependency>
			<groupId>org.apache.commons</groupId>
			<artifactId>commons-lang3</artifactId>
		</dependency>
		<!--redisson must end -->
		<dependency>
			<groupId>org.redisson</groupId>
			<artifactId>redisson-spring-data-21</artifactId>
			<version>3.13.1</version>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-test</artifactId>
			<scope>test</scope>
			<exclusions>
				<exclusion>
					<groupId>org.springframework.boot</groupId>
					<artifactId>spring-boot-starter-logging</artifactId>
				</exclusion>
				<exclusion>
					<groupId>org.slf4j</groupId>
					<artifactId>slf4j-log4j12</artifactId>
				</exclusion>
			</exclusions>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-log4j2</artifactId>
			<exclusions>
				<exclusion>
					<groupId>org.apache.logging.log4j</groupId>
					<artifactId>log4j-core</artifactId>
				</exclusion>
				<exclusion>
					<groupId>org.apache.logging.log4j</groupId>
					<artifactId>log4j-api</artifactId>
				</exclusion>
			</exclusions>
		</dependency>
		<!-- The old version is excluded log4j2 Upgrade to the latest 2.17.1 -->
		<dependency>
			<groupId>org.apache.logging.log4j</groupId>
			<artifactId>log4j-api</artifactId>
			<version>${log4j2.version}</version>
		</dependency>
		<dependency>
			<groupId>org.apache.logging.log4j</groupId>
			<artifactId>log4j-core</artifactId>
			<version>${log4j2.version}</version>
		</dependency>
		<!-- The old version is excluded log4j2 Upgrade to the latest 2.15.0 -->
		<dependency>
			<groupId>org.aspectj</groupId>
			<artifactId>aspectjweaver</artifactId>
		</dependency>
		<dependency>
			<groupId>com.lmax</groupId>
			<artifactId>disruptor</artifactId>
		</dependency>
		<dependency>
			<groupId>com.alibaba</groupId>
			<artifactId>fastjson</artifactId>
		</dependency>
		<dependency>
			<groupId>com.fasterxml.jackson.core</groupId>
			<artifactId>jackson-databind</artifactId>
		</dependency>
		<dependency>
			<groupId>com.google.guava</groupId>
			<artifactId>guava</artifactId>
		</dependency>
		<dependency>
			<groupId>com.alibaba</groupId>
			<artifactId>fastjson</artifactId>
		</dependency>
		<dependency>
			<groupId>com.fasterxml.jackson.core</groupId>
			<artifactId>jackson-databind</artifactId>
		</dependency>

		<!-- General framework of the project -->
		<!-- <dependency> <groupId>org.mk.demo</groupId> <artifactId>redis-common</artifactId> 
			<version>${redis-common.version}</version> </dependency> -->
		<dependency>
			<groupId>org.mk.demo</groupId>
			<artifactId>common-util</artifactId>
			<version>${common-util.version}</version>
		</dependency>
		<dependency>
			<groupId>org.mk.demo</groupId>
			<artifactId>db-common</artifactId>
			<version>${db-common.version}</version>
		</dependency>
	</dependencies>
</project>

SkyPaymentApp.java

/**
 * System project name: org mk.demo. skypayment SkyPaymentApp. java
 * 
 * Feb 1, 2022-11:16:40 PM 2022XX Company - All Rights Reserved
 * 
 */
package org.mk.demo.skypayment;

import org.mybatis.spring.annotation.MapperScan;
import org.redisson.spring.starter.RedissonAutoConfiguration;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.autoconfigure.data.redis.RedisAutoConfiguration;
import org.springframework.boot.autoconfigure.data.redis.RedisRepositoriesAutoConfiguration;
import org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration;
import org.springframework.context.annotation.ComponentScan;

/**
 * 
 * SkyPaymentApp
 * 
 * 
 * Feb 1, 2022 11:16:40 PM
 * 
 * @version 1.0.0
 * 
 */
@SpringBootApplication
@ComponentScan(basePackages = {"org.mk"})
@EnableAutoConfiguration(exclude = {DataSourceAutoConfiguration.class, RedisAutoConfiguration.class,
    RedissonAutoConfiguration.class, RedisRepositoriesAutoConfiguration.class})
@MapperScan("org.mk.demo.skypayment.dao")
public class SkyPaymentApp {

    /**
     * main(Describe the function of this method in one sentence) (describe the applicable conditions of this method here – optional)
     * 
     * @param args
     *            void
     * @exception @since
     *                1.0.0
     */
    public static void main(String[] args) {
        SpringApplication.run(SkyPaymentApp.class);

    }

}

PaymentController.java

/**
 * System project name: org mk.demo. skypayment. controller PaymentController. java
 * 
 * Feb 3, 2022-12:08:59 AM 2022XX Company - All Rights Reserved
 * 
 */
package org.mk.demo.skypayment.controller;

import javax.annotation.Resource;

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.mk.demo.util.response.ResponseBean;
import org.mk.demo.util.response.ResponseCodeEnum;
import org.mk.demo.skypayment.service.Publisher;
import org.mk.demo.skypayment.vo.PaymentBean;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.ResponseBody;
import org.springframework.web.bind.annotation.RestController;

import reactor.core.publisher.Mono;

/**
 * 
 * PaymentController
 * 
 * 
 * Feb 3, 2022 12:08:59 AM
 * 
 * @version 1.0.0
 * 
 */

@RestController
public class PaymentController {
    private Logger logger = LogManager.getLogger(this.getClass());

    @Resource
    private Publisher publisher;

    @PostMapping(value = "/updateStatus", produces = "application/json")
    @ResponseBody
    public Mono<ResponseBean> updateStatus(@RequestBody PaymentBean payment) {
        ResponseBean resp = new ResponseBean();
        try {
            publisher.publishPaymentStatusChange(payment);
            resp = new ResponseBean(ResponseCodeEnum.SUCCESS.getCode(), "success");
        } catch (Exception e) {
            resp = new ResponseBean(ResponseCodeEnum.FAIL.getCode(), "system error");
            logger.error(">>>>>>updateStatus error: " + e.getMessage(), e);
        }
        return Mono.just(resp);
    }
}

Topics: Microservices

Programmer Think

Microservice design guidance - use the "dead letter delay" queue to completely solve the database "deadlock" problem during business timeout compensation

brief introduction

Analog deadlock

preparation

PaymentDao.xml

PaymentService.java

Code to start 5 threads in relevant unit tests

8.47 million pieces of data were created in the database

Then run it. What do we see?

All this comes from being unfamiliar with the principle of batch update (including delete) in the database

Using "delay" queue to solve the design of running batch of business compensation class

application-local.yml

RabbitMqConfig.java

RabbitMqListenerConfig.java

PaymentDao.xml

Publisher.java

Subscriber.java

Run the example of using dead letter queue to realize business compensation

Attachment

parent-pom.xml

pom.xml

SkyPaymentApp.java

PaymentController.java

Hot Topics