abstract

RabbitMQ is a distributed middleware that can be used to realize message communication and service decoupling. It has many typical application scenarios in actual projects, such as asynchronous logging, asynchronous communication decoupling of business service modules, traffic peak shaving and flow restriction in high concurrent rush buying scenarios. In this paper, we will take the typical business scenario in actual projects: "send mail" as an example, Discuss how RabbitMQ can ensure that messages are not lost and 100% successful delivery when sending messages.

Aside ~ in Java technology, this is undoubtedly a high-frequency question

content

• Introduction to overall business process

The business scenario of "sending e-mail" must have been experienced by all partners. When the business requirements are not very strict, the "sending e-mail" can be realized in an asynchronous way. The traditional method is to directly add the annotation @ Async to the method of sending e-mail;

In the era of microservices and distribution, middleware is used to implement more. At this time, MQ comes in handy. At present, the typical products on the market include ActiveMQ, RabbitMQ, Kafka and RocketMQ. Here we mainly focus on RabbitMQ, How can RabbitMQ ensure that messages are not lost and 100% successful delivery during message sending?

We'd better give the answers to this question first. Later, we will use code practice to verify these answers one by one:

(1) "Send confirmation" mode: after the producer sends a message through MQ, MQ needs to feed back the "sent success / failure" to the producer to inform the producer that the message has been delivered successfully. This method can ensure that the message is sent to RabbitMQ correctly

(2) "Consumption confirmation" mode: after listening to the message in the MQ squadron and executing the corresponding business logic, the consumer needs to send a feedback that "the message has been successfully monitored and consumed" to MQ. This method can ensure that the receiver has correctly received and consumed the message. After successful consumption, the message will be removed from the queue

(3) "Avoid repeated message delivery": when the producer produces messages, MQ will generate an MsgId for each message, which can be used as the basis for de duplication (message delivery fails and retransmission) to avoid repeated messages from entering the queue;

(4) "Idempotency guarantee during message consumption": this can be realized by using the characteristics of the business itself, that is, each business entity generally has a unique ID, just like the unique primary key in the database table. When listening for consumption processing, the ID is used as the basis for de duplication;

(5) "Persistence": set the queue, switch and message to the persistence mode to ensure that the message still exists when the MQ service is restarted and recovered during delivery and sending;

(6) "Message consumption retry mechanism": it means that the consumer has an exception in the process of listening, consuming and processing messages, resulting in the failure of business logic. At this time, the "message re-entry queue" mechanism can be started, and the message re-entry queue can be set for N times to retry consumption;

(7) "Message delivery compensation mechanism": it refers to the "delivery failure" of messages during production and delivery, that is, "sending failed". At this time, you can add them to the DB, start the scheduled task, pull those messages that failed to be delivered and re deliver them to the queue. In this way, you can ensure that the messages are not lost and ready to be delivered.

After explaining the answers, let's take the business scenario of "sending mail" as an example and verify the above answers one by one in combination with the actual code practice, as shown in the figure below, as well as the overall implementation flow chart of "RabbitMQ ensures that messages are not lost and 100% delivered successfully":    

• Practice of core code of overall business process

(1) First, you need to create a database table MSG in the database_ Log is used to record the delivery and consumption process of RabbitMQ messages. Its DDL is as follows:

CREATE TABLE `msg_log` (
  `msg_id` varchar(155) NOT NULL DEFAULT '' COMMENT 'Message unique identifier',
  `msg` text COMMENT 'Message body, json format',
  `exchange` varchar(255) NOT NULL DEFAULT '' COMMENT 'Switch',
  `routing_key` varchar(255) NOT NULL DEFAULT '' COMMENT 'Routing key',
  `status` int(11) NOT NULL DEFAULT '0' COMMENT 'state: 0 In delivery, 1 delivery succeeded, 2 delivery failed, 3 consumed',
  `try_count` int(11) NOT NULL DEFAULT '0' COMMENT 'retry count',
  `next_try_time` datetime DEFAULT NULL COMMENT 'Next retry time',
  `create_time` datetime DEFAULT NULL COMMENT 'Creation time',
  `update_time` datetime DEFAULT NULL COMMENT 'Update time',
  PRIMARY KEY (`msg_id`),
  UNIQUE KEY `unq_msg_id` (`msg_id`) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='Message delivery log';

Next, you need to use Mybatis reverse engineering to generate the Entity class Entity, SQL operation interface Mapper and Mapper.xml corresponding to the database table, which will not be posted here. You can generate it yourself (you can also download the corresponding source code at the end of the article and directly copy it for use)

(2) The next step is to create a MailController for sending mail. Its complete source code is as follows:

@RestController
@RequestMapping("mail")
public class MailController {

    @Autowired
    private MailService mailService;

    @Autowired
    private RabbitTemplate rabbitTemplate;

    @Autowired
    private Environment env;

    @Autowired
    private MsgLogMapper msgLogMapper;

    private static final Snowflake snowflake=new Snowflake(3,2);

    //Send mail
    @RequestMapping(value = "send/v2",method = RequestMethod.POST)
    public BaseResponse sendMailV2(@RequestBody @Validated MailDto dto, BindingResult result){
        String checkRes=ValidatorUtil.checkResult(result);
        if (StringUtils.isNotBlank(checkRes)){
            return new BaseResponse(StatusCode.InvalidParams.getCode(),checkRes);
        }
        BaseResponse response=new BaseResponse(StatusCode.Success);
        try {
            MsgLog entity=new MsgLog();
            String msgId=snowflake.nextIdStr();

            entity.setMsgId(msgId);
            entity.setExchange(env.getProperty("mq.email.v2.exchange"));
            entity.setRoutingKey(env.getProperty("mq.email.v2.routing.key"));
            entity.setStatus(Constant.MsgStatus.Sending.getStatus());
            entity.setTryCount(0);
            entity.setCreateTime(DateTime.now().toDate());

            dto.setMsgId(msgId);
            final String msg=new Gson().toJson(dto);

            entity.setMsg(msg);
            //Create and store messages before sending them
            msgLogMapper.insertSelective(entity);

            //send message
            rabbitTemplate.convertAndSend(entity.getExchange(), entity.getRoutingKey(),msg,new CorrelationData(entity.getMsgId()));

        }catch (Exception e){
            response=new BaseResponse(StatusCode.Fail.getCode(),e.getMessage());
        }
        return response;
    }
}

From this code, you can know that before RabbitMQ really sends a message, you need to insert the message into the database and set its status to "being delivered"; At the same time, the "snowflake algorithm" tool needs to be used to generate the global unique ID MsgId of the message and use it as the ID of the message;

For more information, see: http://www.mark-to-win.com/tutorial/51081.html