How to ensure that messages are not lost

1. Message loss caused by network abnormality

1. Do a good job in fault tolerance (try - catch) of the method. Sending messages may fail due to network reasons. After the failure, it will be recorded in the database.
2. Keep a log, and the status of each message sent should be recorded.

   -- Message record table
   create table mq_message (
     message_id char(32) not null,
     content text,
     to_exchange varchar(255) default null,
     routing_key varchar(255) default null,
     class_type varchar(255) default null,
     message_status int(1) default 0 comment '0-newly build,1-has been sent,2-Wrong arrival,3-Arrived',
     create_time datetime default null,
     update_time datetime default null
   ) engine=innodb default charset=utf8mb4;
   

3. Retry. If the message is not sent successfully, scan the database regularly and resend the message that is not sent successfully.

2. When the broker goes down, the messages sent are not persistent, resulting in message loss

Only messages that enter the queue can be persisted. The messages sent out do not enter the queue. At this time, the Broker goes down and the messages will be lost.
Solution: publisher must add a confirmation callback mechanism to ensure that the message successfully reaches the Queue. Publisher adds a callback mechanism. If the message sent from Exchange does not reach the Queue, it will execute a callback returnCallback. At this time, the status of the message in the database will be modified. In case of retry.

3. In the automatic ACK state, the message is lost

If the queue is set to automatic ACK, it may cause message loss. Because in this mode, as long as the consumer receives the message, the queue will remove the message. However, it may happen that after the consumer receives the message, the consumer has an exception and does not have time to process the message, resulting in the loss of the message.
Solution: enable the manual ACK. Only after the message is successfully consumed can it be discarded. In other cases, rejoin the message.
In the project of Spring Boot integrating rabbitmq, messages are automatically ACK by default. It can be found in the application In the YML configuration file, modify the parameters to change the message to manual ACK.

spring:
  rabbitmq:
    listener:
      simple:
        acknowledge-mode: manual

How to ensure that messages are not repeated

View the following code

    @RabbitHandler
    public void handleStockLockedRelease(StockLockedTo to, Message message, Channel channel) throws IOException {
        System.out.println("Receive the message of inventory expiration and unlock inventory");
        try {
            skuService.unlockStock(to);
            channel.basicAck(message.getMessageProperties().getDeliveryTag(), false);
        } catch (Exception e) {
            // Unlocking failed. The message returns to the queue again
            channel.basicReject(message.getMessageProperties().getDeliveryTag(), true);
        }
    }

This is the consumer's code.

1. The message consumption is successful and the transaction has been committed, but the machine goes down when the ACK is received.

In manual ack mode, skuservice unlockStock(to); This line of code executes successfully, but it is executing channel basicAck(message.getMessageProperties(). getDeliveryTag(), false); The machine went down while the code was. In this way, the message is not manually ack. But if you don't get the message of ACK, you will rejoin the team. When the machine returned to normal, I received this message again. Therefore, messages are received repeatedly.
Solution: skuservice unlockStock(to); In this way, the interface called by the code should be designed as idempotent. For example, you can query whether the status of the message is processed according to the id of the message, or query the status of the corresponding business data processing in the database according to the business meaning represented by the message, so as to ensure that the business will not be executed repeatedly.
RabbitMQ each consumed message has a redelivered field. You can judge whether the message has been redelivered according to the value of this field. This can also be used as the basis for further judgment.

Boolean redelivered = message.getMessageProperties().getRedelivered();

2. Message consumption failed. The message will be received again due to the retry mechanism

This repeated message is processed according to the normal process.

How to ensure that messages are not overstocked

1. Message backlog due to insufficient consumer spending power or consumer downtime

1. In the distributed system, more consumers can be online and the consumption capacity of the consumer end can be increased.
2. Launch a special message queue service to take out messages in batches and store them in the database. Take your time offline.

2. The sender sends too many messages, resulting in message backlog

Specific business scenarios can be considered to avoid excessive traffic reaching the message service.