Rocketmqlrocketmq message consumption retry mechanism

Posted by baffled_in_UK on Thu, 10 Feb 2022 03:10:43 +0100

1. Retry of sequential messages

For sequential messages, when the consumer fails to consume the message, the message queue RocketMQ will automatically retry the message continuously (with an interval of 1 second). At this time, the application will be blocked in message consumption. Therefore, when using sequential messages, it is important to ensure that the application can timely monitor and deal with consumption failures to avoid blocking.

2. Retry of unordered messages

For unordered messages (normal, delayed, transaction messages), when the consumer fails to consume the message, the message retry result can be achieved by setting the return status.

The retry of unordered messages only takes effect for the cluster consumption mode; The broadcast mode does not provide failure retry feature, that is, after consumption fails, the failure message will not be retried and new messages will continue to be consumed.

1) Number of retries

Message queue RocketMQ allows each message to be retried up to 16 times by default. The interval between retries is as follows:

Number of retriesTime between last retryNumber of retriesTime between last retry
110 seconds97 minutes
230 seconds108 minutes
31 minute119 minutes
42 minutes1210 minutes
53 minutes1320 minutes
64 minutes1430 Minutes
75 minutes151 hour
86 minutes162 hours

If the message still fails after 16 retries, the message will not be delivered. If the above retry interval is strictly followed, a message will be retried 16 times in the next 4 hours and 46 minutes on the premise of continuous consumption failure. Beyond this time range, the message will not be retried.

Note: no matter how many times a message is retried, the Message ID of these retried messages will not change.

2) Configuration mode

After consumption fails, retry the configuration method

In the cluster consumption mode, if the message consumption fails and the message is expected to be retried later, it needs to be clearly configured in the implementation of the message listener interface (choose one of the three methods):

  • Return to action Recomsumelater (recommended)
  • Return Null
  • Throw exception
public class MessageListenerImpl implements MessageListener {
    @Override
    public Action consume(Message message, ConsumeContext context) {
        //Processing messages
        doConsumeMessage(message);
        if(exception){
        //Method 1: return action Reconsumelater, the message will be retried
        return Action.ReconsumeLater;
        //Method 2: return null and the message will be retried
        return null;
        //Method 3: throw an exception directly and the message will be retried
        throw new RuntimeException("Consumer Message exceotion");
      }
    }
}

After consumption fails, the configuration method will not be retried

In the cluster consumption mode, if the message fails, it is expected that the message will not be retried later. You need to catch the exceptions that may be thrown in the consumption logic and finally return to action Commitmessage, after which this message will not be retried.

public class MessageListenerImpl implements MessageListener {
    @Override
    public Action consume(Message message, ConsumeContext context) {
        try {
            doConsumeMessage(message);
        } catch (Throwable e) {
            //Capture all exceptions in the consumption logic and return action CommitMessage;
            return Action.CommitMessage;
        }
        //The message processing is normal and the action is returned directly CommitMessage;
        return Action.CommitMessage;
    }
}

Note: after integrating SpringBoot, the default is that when the consumer throws an exception, it will automatically retry without manual coding.

Maximum number of retries for custom messages

Message queuing RocketMQ allows the maximum number of retries to be set when the Consumer starts. The retry interval will follow the following policy:

  • If the maximum number of retries is less than or equal to 16, the retry interval is described in the table above.
  • The maximum number of retries is more than 16, and the retry interval of more than 16 times is 2 hours each time.

Native mode:

Properties properties = new Properties();
//The maximum number of message retries configured for the corresponding Group ID is 20
properties.put(PropertyKeyConst.MaxReconsumeTimes,"20");
Consumer consumer =ONSFactory.createConsumer(properties);

Integrate SpringBoot mode:

// Set the maxReconsumeTimes property of the listener, such as 20
@Component
@RocketMQMessageListener(maxReconsumeTimes = 20,topic = "xxx", consumerGroup = "xxx")
public class TestConsumer implements RocketMQListener<String> {}

be careful:

  • The setting of the maximum number of message retries is valid for all Consumer instances under the same Group ID.
  • If MaxReconsumeTimes is set for only one of the two Consumer instances under the same Group ID, the configuration will take effect for both Consumer instances.
  • The configuration takes effect by overwriting, that is, the last started Consumer instance will overwrite the configuration of the previous started instance

Get message retries

After receiving the message, the consumer can obtain the number of retries of the message as follows:

Native mode:

public class MessageListenerImpl implements MessageListener {
    @Override
    public Action consume(Message message, ConsumeContext context) {
        //Number of retries to get the message
        System.out.println(message.getReconsumeTimes());
        return Action.CommitMessage;
    }
}

Integrate SpringBoot mode:

// By implementing the rocketmqlistener < MessageExt > interface, the message is obtained from the MessageExt entity
@Component
@RocketMQMessageListener(topic = "xxx", consumerGroup = "xxx")
public class ExConsumer implements RocketMQListener<MessageExt> {

    @Override
    public void onMessage(MessageExt message) {
        // retry count
        message.getReconsumeTimes();
    }
}

Topics: Java message queue RocketMQ