stay RocketMQ repeated consumption problem | the introduction of idempotent mechanism into the core process of order system In this paper, we discuss the problem of repeated message consumption. A better solution is to use business judgment on the consumption side to ensure the idempotence of the interface, so as to avoid the problem of repeated message consumption.

What we are going to discuss today is the exception in the execution of consumer code. What should we do?

Submit offset manually

First of all, let's look at a piece of code. The Consumer class is a Consumer class. We have registered a listener for it. After processing the message, it will return the status of the message to RocketMQ. If the execution is successful, the status of the message is resume_ SUCCESS.

public class Consumer {
    public static void main(String[] args) throws MQClientException {
        DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("test_consumer");

        // Set NameServer address
        consumer.setNamesrvAddr("");
        // Subscribe to Topic
        consumer.subscribe("TopicTest", "*");
        // This callback interface, receive message
        consumer.registerMessageListener(new MessageListenerConcurrently() {
            @Override
            public ConsumeConcurrentlyStatus consumeMessage(List <MessageExt> list, ConsumeConcurrentlyContext consumeConcurrentlyContext) {
								// Handling of messages, such as coupons, points, etc
								return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
            }
        });
        consumer.start();
    }
}

Draw a diagram to show the process of submitting message status to RocketMQ, as shown in the figure:

What should I do if there is an exception in the messenger's business code?

Let's take a look at the listener part of the consumer's code. It says that if the message processing is successful, the message status will be returned as resume_ Success, it is also possible that there are exceptions in the operation of issuing coupons and points, such as database hang up. What should we do at this time?

consumer.registerMessageListener(new MessageListenerConcurrently() {
            @Override
            public ConsumeConcurrentlyStatus consumeMessage(List <MessageExt> list, ConsumeConcurrentlyContext consumeConcurrentlyContext) {
								// Handling of messages, such as coupons, points, etc
								return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
            }
        });

We can change the code, and the status of the returned message after catching the exception is reconfigure_ Later means to try again later.

// This callback interface, receive message
        consumer.registerMessageListener(new MessageListenerConcurrently() {
            @Override
            public ConsumeConcurrentlyStatus consumeMessage(List <MessageExt> list, ConsumeConcurrentlyContext consumeConcurrentlyContext) {
                try {
                    // Handling of messages, such as coupons, points, etc
                    return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
                } catch (Exception e) {
                    // In case of database downtime and other exceptions, return the status of retry message later
                    return ConsumeConcurrentlyStatus.RECONSUME_LATER;
                }

            }
        });

Retry Queue

At this time, the message will enter the retry queue of RocketMQ.

• For example, the message group name of the consumer is AAAConsumerGroup
• Its retry queue name is called% RETRY%AAAConsumerGroup
• The message in the retry queue will be sent to the consumer again after a period of time. If it still fails to execute normally, it will enter the retry queue again
• By default, if it retries 16 times or fails to execute, the message will enter the dead letter queue from the retry queue

Dead letter queue

• If the message in the retry queue cannot be retried 16 times, it will enter the dead letter queue
• The name of dead letter queue is% DLQ%AAAConsumerGroup
• Messages in the dead letter queue can start a thread in the background, subscribe to% DLQ%AAAConsumerGroup, and keep retrying

summary

This paper introduces the retry queue and dead letter queue of RocketMQ, starting from the exception of the business code of the consumer

1. The normal execution of the code returns the message status as CONSUME_SUCCESS, execute exception, return RECONSUME_LATER
2. Status RECONSUME_LATER's message will enter the retry queue with the name% RETRY% + ConsumerGroupName;
3. If the message is not processed successfully after 16 retries, the message will enter the dead letter queue% DLQ% + ConsumerGroupName;