RocketMQ message sending and message queuing load mechanism

Posted by uchacker11 on Fri, 17 Jan 2020 08:55:27 +0100

After the message producer starts, we can send the message as needed. Before sending the message, we need to obtain the routing information of the subject first. Only after obtaining the information can we know that the message will be sent to the specific Broker node.

Find routing information for a topic

tryToFindTopicPublishlnfo is a way to find routing information for a topic. If the routing information of topic is cached in the producer, if the routing information contains message queue, the routing information will be returned directly. If there is no cache or message queue, the routing information of the topic will be queried from NameServer. If the routing information is not found at last, an exception is thrown: the subject related routing information exception cannot be found.

Private TopicPublishInfo tryToFindTopicPublishInfo(final Stringtopic){
	TopicPublishInfo topicPublishInfo = this.topicPublishInfoTable.get(topic);
	if(null == topicPublishInfo || !topicPublishInfo.ok()){
		this.topicPublishInfoTable.putIfAbsent(topic,newTopicPublishInfo());
		this.mQClientFactory.updateTopicRouteInfoFromNameServer(topic);
		topicPublishInfo=this.topicPublishInfoTable.get(topic);
	}
	
	if(topicPublishInfo.isHaveTopicRouterInfo()||topicPublishInfo.ok()){
		return topicPublishInfo;
	}else{
		this.mQClientFactory.updateTopicRouteInfoFromNameServer(topic,true,this.defaultMQProducer);
		topicPublishInfo = this.topicPublishInfoTable.get(topic);
		returntopicPublishInfo;
	}
}

When sending a message for the first time, there is no route information for caching topic locally. Query NameServer to try to obtain, such as
If the routing information is not found, try to query with the default topic defaultmqproducerlmpl ා createtopickey again. If brokerconfig ා autocreatetopicenable is true, NameServer will return the routing information, and if autoCreateTopicEnab is false, it will throw the topic routing exception that cannot be found.

Message producers update and maintain route cache

The operation of updating and maintaining the route cache by the message producer is in the updateTopicRoutelnfoFromNameServer method of MQClientlnstance:
1. If isDdfault is true, the default subject is used to query. If the route information is queried, the number of read and write queues in the replacement route information is the default number of queues (defaulttopicquenums) of the message producer. If isDefault is false, the parameter topic is used to query. If the route information is not queried, false is returned, indicating that the route information has not changed Change.

	TopicRouteDatatopicRouteData;
	if(isDefault&&defaultMQProducer!=null){
	topicRouteData=this.mQClientAPIImpl.getDefaultTopicRouteInfoFromNameServer(defaultMQProducer.getCreateTopicKey(),
	1000*3);
	if(topicRouteData!=null){
	for(QueueDatadata:topicRouteData.getQueueDatas()){
	intqueueNums=Math.min(defaultMQProducer.getDefaultTopicQueueNums(),data.getReadQueueNums());
	data.setReadQueueNums(queueNums);
	data.setWriteQueueNums(queueNums);
	}
	}
	}else{
	topicRouteData=this.mQClientAPIImpl.getTopicRouteInfoFromNameServer(topic,1000*3);
	}

2. If the routing information is found, compare it with the routing information in the local cache to determine whether the routing information has changed. If not, return false directly.

	TopicRouteDataold=this.topicRouteTable.get(topic);
	booleanchanged=topicRouteDataIsChange(old,topicRouteData);
	if(!changed){
	changed=this.isNeedUpdateTopicRouteInfo(topic);
	}else{
	log.info("thetopic[{}]routeinfochanged,old[{}],new[{}]",topic,old,topicRouteData);
	}

3. Update MQClientlnstance Broker address cache table
4. According to the List in topicproutedata, it is converted to the List of topicPublishInfo, which is implemented in topicproutedata2topicPublishInfo. Then, all messages governed by the MQClientlnstance will be updated to send routing information about the topic.

Loop through the QueueData information of route information. If the queue does not have write permission, continue to traverse the next QueueData; if the brokerData information is found according to the brokerName, and the Master node is not found, traverse the next QueueData; according to the number of write queues, create a MessageQueue according to the topic + sequence number, and fill in the List of topicPublishlnfo

At this point, the routing lookup of message sending is completed.

Published 20 original articles, won praise 11, visited 10000+
Private letter follow