@Override
public void updateBrokerConfig(String brokerAddr,
    Properties properties) throws RemotingConnectException, RemotingSendRequestException,
    RemotingTimeoutException, UnsupportedEncodingException, InterruptedException, MQBrokerException {
    defaultMQAdminExtImpl.updateBrokerConfig(brokerAddr, properties);
}

@Override
public Properties getBrokerConfig(final String brokerAddr) throws RemotingConnectException,
    RemotingSendRequestException, RemotingTimeoutException, UnsupportedEncodingException, InterruptedException, MQBrokerException {
    return defaultMQAdminExtImpl.getBrokerConfig(brokerAddr);
}

@Override
public void createAndUpdateTopicConfig(String addr, TopicConfig config) throws RemotingException, MQBrokerException,
    InterruptedException, MQClientException {
    defaultMQAdminExtImpl.createAndUpdateTopicConfig(addr, config);
}

The next three methods

Update broker configuration

Get the configuration of the broker

Create or upgrade broker configuration

@Override
public void createAndUpdatePlainAccessConfig(String addr,
    PlainAccessConfig config) throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
    defaultMQAdminExtImpl.createAndUpdatePlainAccessConfig(addr, config);
}

@Override public void deletePlainAccessConfig(String addr,
    String accessKey) throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
    defaultMQAdminExtImpl.deletePlainAccessConfig(addr, accessKey);
}

Addition, deletion and modification of ordinary access

@Override public void updateGlobalWhiteAddrConfig(String addr,
    String globalWhiteAddrs) throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
    defaultMQAdminExtImpl.updateGlobalWhiteAddrConfig(addr, globalWhiteAddrs);
}

Update the global whitelist configuration, which is a global whitelist address

@Override public ClusterAclVersionInfo examineBrokerClusterAclVersionInfo(
    String addr) throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
    return defaultMQAdminExtImpl.examineBrokerClusterAclVersionInfo(addr);
}

@Override public AclConfig examineBrokerClusterAclConfig(
    String addr) throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
    return defaultMQAdminExtImpl.examineBrokerClusterAclConfig(addr);
}

Check the acl version information of the broker cluster and check the acl configuration of the broker cluster

This involves the concept of acl

RocketMQ ACL User Guide

ACL is short for access control list, commonly known as access control list. Access control basically involves the concepts of user, resource, permission and role. What objects do the above correspond to in RocketMQ?

• user
  User is the basic element of access control, and it is not difficult to understand. RocketMQ ACL will inevitably introduce the concept of user, that is, it supports user name and password.
• resources
  Resources, objects to be protected. In RocketMQ, topics involved in message sending and consumption groups involved in message consumption should be protected, so they can be abstracted into resources.
• jurisdiction
  Operations that can be performed for resources,
• role
  In RocketMQ, only two roles are defined: administrator or not.

In addition, RocketMQ also supports setting the whitelist according to the client IP.

Back to the code

@Override
public void createAndUpdateSubscriptionGroupConfig(String addr,
    SubscriptionGroupConfig config) throws RemotingException,
    MQBrokerException, InterruptedException, MQClientException {
    defaultMQAdminExtImpl.createAndUpdateSubscriptionGroupConfig(addr, config);
}

@Override
public SubscriptionGroupConfig examineSubscriptionGroupConfig(String addr, String group)
    throws InterruptedException, RemotingException, MQClientException, MQBrokerException {
    return defaultMQAdminExtImpl.examineSubscriptionGroupConfig(addr, group);
}

@Override
public TopicConfig examineTopicConfig(String addr, String topic) throws RemotingException, InterruptedException, MQBrokerException {
    return defaultMQAdminExtImpl.examineTopicConfig(addr, topic);
}

@Override
public TopicStatsTable examineTopicStats(
    String topic) throws RemotingException, MQClientException, InterruptedException,
    MQBrokerException {
    return defaultMQAdminExtImpl.examineTopicStats(topic);
}

Create and update subscription group configuration

View subscription group configuration

Check topic configuration

Check topic status

@Override
public TopicList fetchAllTopicList() throws RemotingException, MQClientException, InterruptedException {
    return this.defaultMQAdminExtImpl.fetchAllTopicList();
}

@Override
public TopicList fetchTopicsByCLuster(
    String clusterName) throws RemotingException, MQClientException, InterruptedException {
    return this.defaultMQAdminExtImpl.fetchTopicsByCLuster(clusterName);
}

@Override
public KVTable fetchBrokerRuntimeStats(
    final String brokerAddr) throws RemotingConnectException, RemotingSendRequestException,
    RemotingTimeoutException, InterruptedException, MQBrokerException {
    return this.defaultMQAdminExtImpl.fetchBrokerRuntimeStats(brokerAddr);
}

Get all topic s

Get all topic s according to the cluster

Gets the status of the broker runtime

Here, let's take a detailed look at how to obtain the topic of the cluster

The parameter passed in here is the cluster name clusterName

public TopicList getTopicsByCluster(final String cluster, final long timeoutMillis)
    throws RemotingException, MQClientException, InterruptedException {
    GetTopicsByClusterRequestHeader requestHeader = new GetTopicsByClusterRequestHeader();
    requestHeader.setCluster(cluster);
    RemotingCommand request = RemotingCommand.createRequestCommand(RequestCode.GET_TOPICS_BY_CLUSTER, requestHeader);

    RemotingCommand response = this.remotingClient.invokeSync(null, request, timeoutMillis);
    assert response != null;
    switch (response.getCode()) {
        case ResponseCode.SUCCESS: {
            byte[] body = response.getBody();
            if (body != null) {
                TopicList topicList = TopicList.decode(body, TopicList.class);
                return topicList;
            }
        }
        default:
            break;
    }

    throw new MQClientException(response.getCode(), response.getRemark());
}

Let's go straight to the bottom and take a look at this method

RemotingCommand request = RemotingCommand.createRequestCommand(RequestCode.GET_TOPICS_BY_CLUSTER, requestHeader);

RemotingCommand response = this.remotingClient.invokeSync(null, request, timeoutMillis);

Mainly these two lines. All topic s of the corresponding cluster will be directly obtained in the body of the result obtained after rpc call

Go and see what asynchronous calls are

You can't find it along the rpc. As expected, it's not as good as the global direct search

org.apache.rocketmq.namesrv.processor.DefaultRequestProcessor#getTopicsByCluster

private RemotingCommand getTopicsByCluster(ChannelHandlerContext ctx,
    RemotingCommand request) throws RemotingCommandException {
    final RemotingCommand response = RemotingCommand.createResponseCommand(null);
    final GetTopicsByClusterRequestHeader requestHeader =
        (GetTopicsByClusterRequestHeader) request.decodeCommandCustomHeader(GetTopicsByClusterRequestHeader.class);

    byte[] body = this.namesrvController.getRouteInfoManager().getTopicsByCluster(requestHeader.getCluster());

    response.setBody(body);
    response.setCode(ResponseCode.SUCCESS);
    response.setRemark(null);
    return response;
}

Finally, I found the implementation of the method here. Take a closer look

We can see the RemotingCommand request passed out

I got it here

byte[] body = this.namesrvController.getRouteInfoManager().getTopicsByCluster(requestHeader.getCluster());

Obviously, this is the most important line

Get the cluster name from the request header, and then

adopt

getTopicsByCluster(requestHeader.getCluster())

Method to get Topics

Note the location of this method

Routing information management in namesrvController RouteInfoManager

The naming here is still very standard

According to the theory we have learned, namesrv stores "names", including broker information ip. In short, all related to ip, name, etc

Go to namesrv

namesrv is actually something like eurake zk

Total routing

look down

getTopicsByCluster(requestHeader.getCluster())

This method

public byte[] getTopicsByCluster(String cluster) {
    TopicList topicList = new TopicList();
    try {
        try {
            this.lock.readLock().lockInterruptibly();
            Set<String> brokerNameSet = this.clusterAddrTable.get(cluster);
            for (String brokerName : brokerNameSet) {
                Iterator<Entry<String, List<QueueData>>> topicTableIt =
                    this.topicQueueTable.entrySet().iterator();
                while (topicTableIt.hasNext()) {
                    Entry<String, List<QueueData>> topicEntry = topicTableIt.next();
                    String topic = topicEntry.getKey();
                    List<QueueData> queueDatas = topicEntry.getValue();
                    for (QueueData queueData : queueDatas) {
                        if (brokerName.equals(queueData.getBrokerName())) {
                            topicList.getTopicList().add(topic);
                            break;
                        }
                    }
                }
            }
        } finally {
            this.lock.readLock().unlock();
        }
    } catch (Exception e) {
        log.error("getAllTopicList Exception", e);
    }

    return topicList.encode();

Sure enough, we see that there is no mystery at the bottom, that is, we store all the information in memory~

The first is a cluster address table. The clusterAddrTable stores the mapping of cluster name and broker name

Then there is a topic queue table, which stores the mapping between topic name and queue. It can be seen that the relationship is one to many

If the broker name of the queue is consistent with the broker name obtained from the cluster name

Then the topic corresponding to the queue is the topic corresponding to the cluster name

To sum up:

The cluster name cluster actually has a one to many relationship with broker

private HashMap<String/* clusterName */, Set<String/* brokerName */>> clusterAddrTable;

@Override
public ConsumeStats examineConsumeStats(
    String consumerGroup) throws RemotingException, MQClientException, InterruptedException,
    MQBrokerException {
    return examineConsumeStats(consumerGroup, null);
}

@Override
public ConsumeStats examineConsumeStats(String consumerGroup,
    String topic) throws RemotingException, MQClientException,
    InterruptedException, MQBrokerException {
    return defaultMQAdminExtImpl.examineConsumeStats(consumerGroup, topic);
}

@Override
public ClusterInfo examineBrokerClusterInfo() throws InterruptedException, RemotingConnectException, RemotingTimeoutException,
    RemotingSendRequestException, MQBrokerException {
    return defaultMQAdminExtImpl.examineBrokerClusterInfo();
}

Keep going

Check consumption status (Progress) and broker cluster information

org.apache.rocketmq.common.admin.ConsumeStats

Here's what is the consumption status (Progress)

public class ConsumeStats extends RemotingSerializable {
    private HashMap<MessageQueue, OffsetWrapper> offsetTable = new HashMap<MessageQueue, OffsetWrapper>();
    private double consumeTps = 0;

There is a schedule offsetTable and a consumption tps consumeTps

MessageQueue is the basic information of message queue

private String topic;
private String brokerName;
private int queueId;

org.apache.rocketmq.common.admin.OffsetWrapper

Information of offsetWrapper:

private long brokerOffset;
private long consumerOffset;

private long lastTimestamp;

The progress of the broker, the progress of the consumer, and the timestamp of the last update

@Override
public ClusterInfo examineBrokerClusterInfo() throws InterruptedException, RemotingConnectException, RemotingTimeoutException,
    RemotingSendRequestException, MQBrokerException {
    return defaultMQAdminExtImpl.examineBrokerClusterInfo();
}

@Override
public TopicRouteData examineTopicRouteInfo(
    String topic) throws RemotingException, MQClientException, InterruptedException {
    return defaultMQAdminExtImpl.examineTopicRouteInfo(topic);
}

@Override
public ConsumerConnection examineConsumerConnectionInfo(
    String consumerGroup) throws InterruptedException, MQBrokerException,
    RemotingException, MQClientException {
    return defaultMQAdminExtImpl.examineConsumerConnectionInfo(consumerGroup);
}

Check the broker cluster information, topic routing information, and consumer connection information

Look at the cluster information. What is ClusterInfo

org.apache.rocketmq.common.protocol.body.ClusterInfo

Take a look at the mysterious rocketmq cluster:

public class ClusterInfo extends RemotingSerializable {
    private HashMap<String/* brokerName */, BrokerData> brokerAddrTable;
    private HashMap<String/* clusterName */, Set<String/* brokerName */>> clusterAddrTable;

A table of breaker name brokerName and breaker data BrokerData

A table of cluster name clusterName and circuit breaker name brokerName

In fact, one cluster name corresponds to multiple brokers, and then the brokers correspond to their own data