Sequential consumption of RocketMQ MessageListenerOrderly()

Posted by gvanaco on Thu, 17 Feb 2022 07:29:29 +0100

consumer.registerMessageListener(new MessageListenerOrderly() {
            @Override
            public ConsumeOrderlyStatus consumeMessage(List<MessageExt> msgs, ConsumeOrderlyContext context) {
                for (MessageExt msg : msgs) {
                    System.out.println(new String(msg.getBody())+" Thread:"+Thread.currentThread()+" QueueID:"+msg.getQueueId());
                }
                return ConsumeOrderlyStatus.SUCCESS;
            }
        });

When using MessageListenerOrderly(), it is found that only when the consumer starts for the first time, it uses a single thread to consume sequentially
However, single thread is no longer used, which is contrary to the source code annotation (One queue by one thread)

After Baidu, in Sequential messages of RocketMQ (sequential consumption) The answer is found in the article

In fact, each consumer uses thread pool to realize multi-threaded consumption, that is, the consumer is multi-threaded consumption. Although MessageListenerOrderly is called the ordered consumption mode, it still uses the thread pool to consume messages.

Messagelistenercurrently refers to submitting new messages to the thread pool for consumption after pulling them, while MessageListenerOrderly ensures that only one thread can consume data on a queue at the same time by adding distributed locks and local locks.

Locking mechanism of MessageListenerOrderly:

1. When pulling messages from a queue, consumers first apply for a queue lock from the Broker server. If the lock is applied, they pull the message. Otherwise, they give up message pulling and try again in the next queue load cycle (20s). This lock enables a MessageQueue to be consumed by only one consuming client at a time, so as to prevent repeated consumption of messages due to queue load balancing.

2. Assuming that the consumer has successfully locked the messageQueue, it will start to pull the message. After pulling the message, it will also be submitted to the thread pool on the consumer side for consumption. However, before local consumption, the lock object corresponding to the messageQueue will be obtained first, and each messageQueue corresponds to a lock object. After obtaining the lock object, the synchronized blocking application thread level exclusive lock will be used. This lock enables messages from the same messageQueue to be consumed by only one thread in a consuming client at a local time.

3. After the synchronized lock is successfully added locally, it will also be judged that if it is in the broadcast mode, it will be consumed directly. If it is in the cluster mode, it will be judged that if the messagequeue is not locked or the lock expires (30000ms by default), try to apply to the Broker for locking the messagequeue again after a delay of 100ms, and resubmit the consumption request after successful locking.

    public void lockAll() {
        HashMap<String, Set<MessageQueue>> brokerMqs = this.buildProcessQueueTableByBrokerName();

        Iterator<Entry<String, Set<MessageQueue>>> it = brokerMqs.entrySet().iterator();
        while (it.hasNext()) {
            Entry<String, Set<MessageQueue>> entry = it.next();
            final String brokerName = entry.getKey();
            final Set<MessageQueue> mqs = entry.getValue();

            if (mqs.isEmpty())
                continue;

            FindBrokerResult findBrokerResult = this.mQClientFactory.findBrokerAddressInSubscribe(brokerName, MixAll.MASTER_ID, true);
            if (findBrokerResult != null) {
                LockBatchRequestBody requestBody = new LockBatchRequestBody();
                requestBody.setConsumerGroup(this.consumerGroup);
                requestBody.setClientId(this.mQClientFactory.getClientId());
                requestBody.setMqSet(mqs);

                try {
                    Set<MessageQueue> lockOKMQSet =
                        this.mQClientFactory.getMQClientAPIImpl().lockBatchMQ(findBrokerResult.getBrokerAddr(), requestBody, 1000);

                    for (MessageQueue mq : lockOKMQSet) {
                        ProcessQueue processQueue = this.processQueueTable.get(mq);
                        if (processQueue != null) {
                            if (!processQueue.isLocked()) {
                                log.info("the message queue locked OK, Group: {} {}", this.consumerGroup, mq);
                            }

                            processQueue.setLocked(true);
                            processQueue.setLastLockTimestamp(System.currentTimeMillis());
                        }
                    }
                    for (MessageQueue mq : mqs) {
                        if (!lockOKMQSet.contains(mq)) {
                            ProcessQueue processQueue = this.processQueueTable.get(mq);
                            if (processQueue != null) {
                                processQueue.setLocked(false);
                                log.warn("the message queue locked Failed, Group: {} {}", this.consumerGroup, mq);
                            }
                        }
                    }
                } catch (Exception e) {
                    log.error("lockBatchMQ exception, " + mqs, e);
                }
            }
        }
    }

Topics: Java Back-end message queue RocketMQ