I also told you about the rules of RocketMQ read-write separation, but you may ask, how does the consumption progress between the master and slave servers keep pace? Let me give you an answer.

If the consumer consumption mode is different, there will be different ways to save it. The message consumption progress of the consumer side is saved to the OffsetStore, which has two implementation classes:

org.apache.rocketmq.client.consumer.store.LocalFileOffsetStore // Local consumption progress saving implementation
org.apache.rocketmq.client.consumer.store.RemoteBrokerOffsetStore // Implementation of saving progress of remote consumption

Among them, if the consumption is in broadcast mode, the consumption progress of messages is saved locally. If it is in cluster consumption mode, the consumption progress of messages is saved to Broker. However, whether it is saved locally or saved to Broker, consumers will keep a cache locally. Let's take a look at how the cache of message consumption progress is saved in cluster consumption mode.

org.apache.rocketmq.client.consumer.store.RemoteBrokerOffsetStore#updateOffset:

public void updateOffset(MessageQueue mq, long offset, boolean increaseOnly) {
  if (mq != null) {
    AtomicLong offsetOld = this.offsetTable.get(mq);
    if (null == offsetOld) {
      offsetOld = this.offsetTable.putIfAbsent(mq, new AtomicLong(offset));
    }

    if (null != offsetOld) {
      if (increaseOnly) {
        MixAll.compareAndIncreaseOnly(offsetOld, offset);
      } else {
        offsetOld.set(offset);
      }
    }
  }
}

After the message is consumed, the messager will call the above methods to put the consumption progress into the offsetTable cache. When the Rebalance load is redistributed to generate the PullRequest object, the messager will call the remoteborkerofsetstore.readoffset method to take the corresponding consumption progress cache value from the offsetTable cache, and then put the value into the PullRequest object. Next, when the message is pulled. It is very important to send the message consumption progress cache to the Broker side, so let's continue to see the processing logic of the Broker side.

When sorting out the Broker startup process, it is found that a scheduled task will be started when the Broker starts:

org.apache.rocketmq.broker.BrokerController#initialize:

this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {

    @Override
    public void run() {
        try {
            BrokerController.this.slaveSynchronize.syncAll();
        } catch (Throwable e) {
            log.error("ScheduledTask syncAll slave exception", e);
        }
    }
}, 1000 * 10, 1000 * 60, TimeUnit.MILLISECONDS);

If the Broker is a slave, the above scheduled tasks are enabled.

org.apache.rocketmq.broker.slave.SlaveSynchronize#syncAll:

public void syncAll() {
  this.syncTopicConfig();
  this.syncConsumerOffset();
  this.syncDelayOffset();
  this.syncSubscriptionGroupConfig();
}

When the master server is not down, the slave server will synchronize the message consumption progress and other information from the master server at regular intervals. Now the problem comes. Because this synchronization is unilateral, that is, only the master server will be synchronized from the slave server. If the master server is down, the consumer switches to pull the message from the slave server for consumption. If the master server starts later Now, the server is synchronizing the offset that has been consumed. Isn't that causing synchronous consumption?

In fact, when a consumer pulls a message, if there is a consumption progress in the consumer's cache, the consumer will update the message consumption progress to the Broker, so even if the main server is hung, after it is restarted, the consumer's consumption progress will not be lost, and the message consumption progress of the main server will still be updated, so that the consumer and the main server only hang one of the devices. Messages will not be consumed again. The specific code logic is as follows:

org.apache.rocketmq.broker.processor.PullMessageProcessor#processRequest:

boolean storeOffsetEnable = brokerAllowSuspend;
storeOffsetEnable = storeOffsetEnable && hasCommitOffsetFlag;
storeOffsetEnable = storeOffsetEnable
    && this.brokerController.getMessageStoreConfig().getBrokerRole() != BrokerRole.SLAVE;
if (storeOffsetEnable) {
 this.brokerController.getConsumerOffsetManager().commitOffset(RemotingHelper.parseChannelRemoteAddr(channel), requestHeader.getConsumerGroup(), requestHeader.getTopic(), requestHeader.getQueueId(), requestHeader.getCommitOffset());
}

Broker allowsuspend indicates whether the broker is allowed to suspend. The default value is true. hasCommitOffsetFlag indicates whether the message consumption progress is cached by the interest consumer in memory. From the code logic, if the broker is the main server, and both broker allowsuspend and hasCommitOffsetFlag are true, the consumer consumption progress will be updated to the local.