RocketMQ Dual Primary Double Slave Cluster Setup

Posted by somedude on Thu, 10 Feb 2022 10:14:19 +0100

1 Introduction to roles

  • Producer: The sender of the message; Example: Sender
  • Consumer: Message recipient; Example: Recipient
  • Broker: Stay and transfer messages; Example: Post Office
  • NameServer: Manage Broker; Examples: the administrative bodies of various post offices
  • Topic: Differentiate the types of messages; One sender can send a message to one or more Topics; Recipients of a message can subscribe to one or more Topic messages
  • Message Queue: Equivalent to a partition of Topic; For sending and receiving messages in parallel

2 Cluster Setup

2.1 Cluster Characteristics

  • NameServer is a virtually stateless node that can be deployed in a cluster without any information synchronization between nodes.

  • Broker deployment is relatively complex. Broker is divided into Master and Slave. A Master can correspond to multiple Slaves, but a Slave can only correspond to one Master. The corresponding relationship between Master and Slave is defined by specifying the same BrokerName, different BrokerId s, 0 for Master and non-0 for Slave. Master can also deploy multiple. Each broker establishes a long connection to all nodes in the NameServer cluster and periodically registers Topic information to all NameServers.

  • Producer establishes a long connection with one of the nodes in the NameServer cluster (randomly selected), periodically fetches Topic routing information from NameServer, establishes a long connection to the Master providing Topic services, and periodically sends a heartbeat to the Master. Producer is completely stateless and can be deployed in clusters.

  • Consumer establishes a long connection with one of the nodes in the NameServer cluster (randomly selected), periodically fetches Topic routing information from NameServer, establishes a long connection to Master and Slave providing Topic services, and periodically sends heartbeats to Master and Slave. Consumers can subscribe to messages from either Master or Slave, and the subscription rules are determined by the Broker configuration.

2.2 Cluster Mode

1) Single Master Mode

This is a risky approach that will make the entire service unavailable once the Broker restarts or goes down. Online environments are not recommended and can be used for local testing.

2) Multiple Master Mode

A cluster has no Slave and all Masters, such as 2 Masters or 3 Masters. The advantages and disadvantages of this pattern are as follows:

  • Advantages: Simple configuration, single Master downtime or restart maintenance has no effect on the application. When the disk is configured as RAID10, the message will not be lost even if the machine is not restored when the disk is configured as RAID10, because RAID10 disk is very reliable (asynchronous brush disc loses a few messages, synchronous brush disc does not lose one), and the performance is the highest;
  • Disadvantages: During downtime of a single machine, messages that are not consumed on this machine cannot be subscribed until the machine is restored, and message real-time will be affected.

3) Multiple Master Multiple Slave Mode (Asynchronous)

Each Master configures a Slave with multiple pairs of Master-Slave s, HA replicates asynchronously, and primary has a short message delay (in milliseconds). The advantages and disadvantages of this mode are as follows:

  • Advantages: Even if the disk is damaged, messages are lost very little, and message real-time is not affected, and consumers can still consume from Slave after Master is down, and this process is transparent to applications, does not require manual intervention, and performs almost the same as in multiple Master mode;
  • Disadvantages: Master downtime, small amounts of messages will be lost if the disk is damaged.

4) Multiple Master Multiple Slave Mode (Synchronization)

Each Master is configured with a Slave, with multiple pairs of Master-Slave s. HA uses synchronous dual writing, that is, only if the primary and standby are successfully written, will it return success to the application. The advantages and disadvantages of this mode are as follows:

  • Advantages: There is no single point of failure for both data and services. In case of Master downtime, there is no delay for messages, and service availability and data availability are very high.
  • Disadvantages: performance is slightly lower than asynchronous replication mode (about 10 percent lower), RT to send a single message is slightly higher, and the standby cannot automatically switch to the host after the primary node is down.

3 Double Primary and Double Slave Clusters

3.1 Overall Architecture

Message High Availability with 2master-2slave

3.2 Cluster Workflow

  1. Start NameServer, and when NameServer gets up, listen on the port and wait for Broker, Producer, and Consumer to connect, which is equivalent to a routing control center.
  2. The broker starts, maintains a long connection to all NameServers, and periodically sends a heartbeat packet. The heartbeat package contains the current Broker information (IP+port, etc.) and stores all Topic information. After successful registration, Topic maps to Broker in the NameServer cluster.
  3. Before sending and receiving a message, create a Topic, specify which brokers the Topic will store, or automatically create a Topic when sending a message.
  4. Producer sends a message, makes a long connection to one of the NameServer clusters at startup, obtains from NameServer which brokers are currently sent on Topic, polls to select a queue from the queue list, and then makes a long connection to the Broker where the queue is located to send messages to the Broker.
  5. Consumer is similar to Producer in that it establishes a long connection with one of the NameServer s, gets which brokers are on the Topic currently subscribed to, and then establishes a connection channel directly with the Broker to start consuming messages.

3.3 Server Environment

Sequence NumberIProleArchitecture mode
1192.168.2.12nameserver,brokerserverMaster1,Slave2
2192.168.2.13nameserver,brokerserverMaster2,Slave1

3.4 Host Add Information

vim /etc/hosts
  •  

The configuration is as follows:

# nameserver
192.168.2.12 rocketmq-nameserver1
192.168.2.13 rocketmq-nameserver2
# broker
192.168.2.12 rocketmq-master1
192.168.2.12 rocketmq-slave2
192.168.2.13 rocketmq-master2
192.168.2.13 rocketmq-slave1
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  

Restart network card after configuration is complete

systemctl restart network
  •  

3.5 Firewall Configuration

Hosts need remote access to the virtual machine's rocketmq services and web services, open related port numbers, and simply, roughly, close the firewall directly

bash

# Close Firewall
systemctl stop firewalld.service 
# View the status of your firewall
firewall-cmd --state 
# Disable firewall startup
systemctl disable firewalld.service
  •  
  •  
  •  
  •  
  •  
  •  

Or, for security reasons, only certain port numbers are open, RocketMQ uses three ports by default: 9876, 10911, 11011. If the firewall is not closed, then the firewall must open these ports:

  • nameserver uses port 9876 by default
  • master uses port 10911 by default
  • slave uses port 11011 by default

Execute the following commands:

# Open name server default port
firewall-cmd --remove-port=9876/tcp --permanent
# Open master default port
firewall-cmd --remove-port=10911/tcp --permanent
# Open slave default port (current cluster mode is not open)
firewall-cmd --remove-port=11011/tcp --permanent 
# service iptables restart
firewall-cmd --reload
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  

3.6 Configuration of environment variables

vim /etc/profile
  •  

Add the following command at the end of the profile file

#set rocketmq This directory is the rocketmq binary package decompression directory
ROCKETMQ_HOME=/usr/local/rocketmq
PATH=$PATH:$ROCKETMQ_HOME/bin
export ROCKETMQ_HOME PATH
  •  
  •  
  •  
  •  

Input: wq! Save and exit, and make the configuration take effect immediately:

bash

source /etc/profile
  •  

3.7 Create message store path

mkdir /usr/local/rocketmq/store
mkdir /usr/local/rocketmq/store/commitlog
mkdir /usr/local/rocketmq/store/consumequeue
mkdir /usr/local/rocketmq/store/index
  •  
  •  
  •  
  •  

3.8 broker Profile

1)master1

Server: 192.168.2.12

vim /usr/local/rocketmq/conf/2m-2s-sync/new-broker-a.properties
  •  

Modify the configuration as follows:

#The name of the cluster to which you belong
brokerClusterName=rocketmq-cluster
#broker name, note that different profiles here fill in different
brokerName=broker-a
#0 for Master, >0 for Slave
brokerId=0
#Name Server address, semicolon-separated
namesrvAddr=rocketmq-nameserver1:9876;rocketmq-nameserver2:9876
#When sending a message, automatically create a topic that does not exist on the server, default number of queues created
defaultTopicQueueNums=4
#Whether Broker is allowed to automatically create Topic, it is recommended to turn it on offline and turn it off online
autoCreateTopicEnable=true
#Whether Broker is allowed to automatically create subscription groups, it is recommended to open offline and close Online
autoCreateSubscriptionGroup=true
#Broker's listening port for external services
listenPort=10911
#Delete file point in time, default 4 am
deleteWhen=04
#File retention time, default 48 hours
fileReservedTime=120
#commitLog default size of 1G per file
mapedFileSizeCommitLog=1073741824
#ConsumeQueue saves 30W bars per file by default, adjusting to business conditions
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#Detect physical file disk space
diskMaxUsedSpaceRatio=88
#Storage Path
storePathRootDir=/usr/local/rocketmq/store
#commitLog storage path
storePathCommitLog=/usr/local/rocketmq/store/commitlog
#Consumer Queue Storage Path Storage Path
storePathConsumeQueue=/usr/local/rocketmq/store/consumequeue
#Message Index Storage Path
storePathIndex=/usr/local/rocketmq/store/index
#checkpoint file storage path
storeCheckpoint=/usr/local/rocketmq/store/checkpoint
#abort file storage path
abortFile=/usr/local/rocketmq/store/abort
#Limited message size
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
#Broker's role
#- ASYNC_MASTER Asynchronous Replication Master
#- SYNC_MASTER Synchronized Double Write Master
#- SLAVE
brokerRole=SYNC_MASTER
#Brush disc mode
#- ASYNC_FLUSH asynchronous brush disc
#- SYNC_FLUSH synchronous brush disc
flushDiskType=SYNC_FLUSH
#checkTransactionMessageEnable=false
#Number of messaging thread pools
#sendMessageThreadPoolNums=128
#Number of pull message thread pools
#pullMessageThreadPoolNums=128
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  

2)slave2

Server: 192.168.2.12

vim /usr/local/rocketmq/conf/2m-2s-sync/new-broker-b-s.properties
  •  

Modify the configuration as follows:

#The name of the cluster to which you belong
brokerClusterName=rocketmq-cluster
#broker name, note that different profiles here fill in different
brokerName=broker-b
#0 for Master, >0 for Slave
brokerId=1
#Name Server address, semicolon-separated
namesrvAddr=rocketmq-nameserver1:9876;rocketmq-nameserver2:9876
#When sending a message, automatically create a topic that does not exist on the server, default number of queues created
defaultTopicQueueNums=4
#Whether Broker is allowed to automatically create Topic, it is recommended to turn it on offline and turn it off online
autoCreateTopicEnable=true
#Whether Broker is allowed to automatically create subscription groups, it is recommended to open offline and close Online
autoCreateSubscriptionGroup=true
#Broker's listening port for external services
listenPort=11011
#Delete file point in time, default 4 am
deleteWhen=04
#File retention time, default 48 hours
fileReservedTime=120
#commitLog default size of 1G per file
mapedFileSizeCommitLog=1073741824
#ConsumeQueue saves 30W bars per file by default, adjusting to business conditions
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#Detect physical file disk space
diskMaxUsedSpaceRatio=88
#Storage Path
storePathRootDir=/usr/local/rocketmq/store
#commitLog storage path
storePathCommitLog=/usr/local/rocketmq/store/commitlog
#Consumer Queue Storage Path Storage Path
storePathConsumeQueue=/usr/local/rocketmq/store/consumequeue
#Message Index Storage Path
storePathIndex=/usr/local/rocketmq/store/index
#checkpoint file storage path
storeCheckpoint=/usr/local/rocketmq/store/checkpoint
#abort file storage path
abortFile=/usr/local/rocketmq/store/abort
#Limited message size
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
#Broker's role
#- ASYNC_MASTER Asynchronous Replication Master
#- SYNC_MASTER Synchronized Double Write Master
#- SLAVE
brokerRole=SLAVE
#Brush disc mode
#- ASYNC_FLUSH asynchronous brush disc
#- SYNC_FLUSH synchronous brush disc
flushDiskType=ASYNC_FLUSH
#checkTransactionMessageEnable=false
#Number of messaging thread pools
#sendMessageThreadPoolNums=128
#Number of pull message thread pools
#pullMessageThreadPoolNums=128
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  

3)master2

Server: 192.168.2.13

vim /usr/local/rocketmq/conf/2m-2s-sync/new-broker-b.properties
  •  

Modify the configuration as follows:

#The name of the cluster to which you belong
brokerClusterName=rocketmq-cluster
#broker name, note that different profiles here fill in different
brokerName=broker-b
#0 for Master, >0 for Slave
brokerId=0
#Name Server address, semicolon-separated
namesrvAddr=rocketmq-nameserver1:9876;rocketmq-nameserver2:9876
#When sending a message, automatically create a topic that does not exist on the server, default number of queues created
defaultTopicQueueNums=4
#Whether Broker is allowed to automatically create Topic, it is recommended to turn it on offline and turn it off online
autoCreateTopicEnable=true
#Whether Broker is allowed to automatically create subscription groups, it is recommended to open offline and close Online
autoCreateSubscriptionGroup=true
#Broker's listening port for external services
listenPort=10911
#Delete file point in time, default 4 am
deleteWhen=04
#File retention time, default 48 hours
fileReservedTime=120
#commitLog default size of 1G per file
mapedFileSizeCommitLog=1073741824
#ConsumeQueue saves 30W bars per file by default, adjusting to business conditions
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#Detect physical file disk space
diskMaxUsedSpaceRatio=88
#Storage Path
storePathRootDir=/usr/local/rocketmq/store
#commitLog storage path
storePathCommitLog=/usr/local/rocketmq/store/commitlog
#Consumer Queue Storage Path Storage Path
storePathConsumeQueue=/usr/local/rocketmq/store/consumequeue
#Message Index Storage Path
storePathIndex=/usr/local/rocketmq/store/index
#checkpoint file storage path
storeCheckpoint=/usr/local/rocketmq/store/checkpoint
#abort file storage path
abortFile=/usr/local/rocketmq/store/abort
#Limited message size
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
#Broker's role
#- ASYNC_MASTER Asynchronous Replication Master
#- SYNC_MASTER Synchronized Double Write Master
#- SLAVE
brokerRole=SYNC_MASTER
#Brush disc mode
#- ASYNC_FLUSH asynchronous brush disc
#- SYNC_FLUSH synchronous brush disc
flushDiskType=SYNC_FLUSH
#checkTransactionMessageEnable=false
#Number of messaging thread pools
#sendMessageThreadPoolNums=128
#Number of pull message thread pools
#pullMessageThreadPoolNums=128
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  

4)slave1

Server: 192.168.2.13

vim /usr/local/rocketmq/conf/2m-2s-sync/new-broker-a-s.properties
  •  

Modify the configuration as follows:

#The name of the cluster to which you belong
brokerClusterName=rocketmq-cluster
#broker name, note that different profiles here fill in different
brokerName=broker-a
#0 for Master, >0 for Slave
brokerId=1
#Name Server address, semicolon-separated
namesrvAddr=rocketmq-nameserver1:9876;rocketmq-nameserver2:9876
#When sending a message, automatically create a topic that does not exist on the server, default number of queues created
defaultTopicQueueNums=4
#Whether Broker is allowed to automatically create Topic, it is recommended to turn it on offline and turn it off online
autoCreateTopicEnable=true
#Whether Broker is allowed to automatically create subscription groups, it is recommended to open offline and close Online
autoCreateSubscriptionGroup=true
#Broker's listening port for external services
listenPort=11011
#Delete file point in time, default 4 am
deleteWhen=04
#File retention time, default 48 hours
fileReservedTime=120
#commitLog default size of 1G per file
mapedFileSizeCommitLog=1073741824
#ConsumeQueue saves 30W bars per file by default, adjusting to business conditions
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#Detect physical file disk space
diskMaxUsedSpaceRatio=88
#Storage Path
storePathRootDir=/usr/local/rocketmq/store
#commitLog storage path
storePathCommitLog=/usr/local/rocketmq/store/commitlog
#Consumer Queue Storage Path Storage Path
storePathConsumeQueue=/usr/local/rocketmq/store/consumequeue
#Message Index Storage Path
storePathIndex=/usr/local/rocketmq/store/index
#checkpoint file storage path
storeCheckpoint=/usr/local/rocketmq/store/checkpoint
#abort file storage path
abortFile=/usr/local/rocketmq/store/abort
#Limited message size
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
#Broker's role
#- ASYNC_MASTER Asynchronous Replication Master
#- SYNC_MASTER Synchronized Double Write Master
#- SLAVE
brokerRole=SLAVE
#Brush disc mode
#- ASYNC_FLUSH asynchronous brush disc
#- SYNC_FLUSH synchronous brush disc
flushDiskType=ASYNC_FLUSH
#checkTransactionMessageEnable=false
#Number of messaging thread pools
#sendMessageThreadPoolNums=128
#Number of pull message thread pools
#pullMessageThreadPoolNums=128
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  

3.9 Modify startup script file

1)runbroker.sh

sh

vi /usr/local/rocketmq/bin/runbroker.sh
  •  

JVM parameters need to be adjusted appropriately according to memory size:

bash

#===================================================
# Development Environment Configuration JVM Configuration
JAVA_OPT="${JAVA_OPT} -server -Xms256m -Xmx256m -Xmn128m"
  •  
  •  
  •  

2)runserver.sh

vim /usr/local/rocketmq/bin/runserver.sh
  •  
JAVA_OPT="${JAVA_OPT} -server -Xms256m -Xmx256m -Xmn128m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m"
  •  

3.10 Service Startup

1) Start the NameServer cluster

NameServer was started in 192.168.2.12 and 192.168.2.13, respectively.

bash

cd /usr/local/rocketmq/bin
nohup sh mqnamesrv &
  •  
  •  

2) Start Broker Cluster

  • Start master 1 and slave2 on 192.168.2.12

master1:

cd /usr/local/rocketmq/bin
nohup sh mqbroker -c /usr/local/rocketmq/conf/2m-2s-sync/new-broker-a.properties &
  •  
  •  

slave2:

cd /usr/local/rocketmq/bin
nohup sh mqbroker -c /usr/local/rocketmq/conf/2m-2s-sync/new-broker-b-s.properties &
  •  
  •  

  • Start Master 2 and slave2 on 192.168.2.13

master2

cd /usr/local/rocketmq/bin
nohup sh mqbroker -c /usr/local/rocketmq/conf/2m-2s-sync/new-broker-b.properties &
  •  
  •  

slave1

cd /usr/local/rocketmq/bin
nohup sh mqbroker -c /usr/local/rocketmq/conf/2m-2s-sync/new-broker-a-s.properties &
  •  
  •  

3.11 View process status

View startup process via JPS after startup


3.12 View Log

# View the nameServer log
tail -500f ~/logs/rocketmqlogs/namesrv.log
# View broker log
tail -500f ~/logs/rocketmqlogs/broker.log
  •  
  •  
  •  
  •  

4. mqadmin management tools

4.1 Usage

Enter the RocketMQ installation location and execute in the bin directory. / mqadmin {command} {args}

4.2 Introduction to Commands

1) Topic correlation

NameMeaningCommand OptionsExplain
updateTopicCreate Update Topic Configuration-bBroker address, indicating the broker where topic is located, supports only a single broker with ip:port address
-cCluster name, indicating the cluster in which topic resides (clusters can be queried through a clusterList)
-h-Print Help
-nNameServer service address, format ip:port
-pSpecify read and write permissions for the new top (W=2|R=4|WR=6)
-rNumber of readable queues (default is 8)
-wNumber of writable queues (default is 8)
-ttopic name (name can only use the character ^[a-zA-Z0-9_-]+$)
deleteTopicDelete Topic-cCluster name indicating the deletion of a topic under a cluster (the cluster can be queried through the cluster list)
-hPrint Help
-nNameServer service address, format ip:port
-ttopic name (name can only use the character ^[a-zA-Z0-9_-]+$)
topicListView Topic List Information-hPrint Help
-cDo not configure -c to return only the topic list, increase-c to return clusterName, topic, consumerGroup information, that is, the cluster to which topic belongs and the subscription relationship, with no parameters
-nNameServer service address, format ip:port
topicRouteView Topic Routing Information-ttopic name
-hPrint Help
-nNameServer service address, format ip:port
topicStatusView Topic Message Queue offset-ttopic name
-hPrint Help
-nNameServer service address, format ip:port
topicClusterListView the list of clusters Topic is in-ttopic name
-hPrint Help
-nNameServer service address, format ip:port
updateTopicPermUpdate Topic Read and Write Permissions-ttopic name
-hPrint Help
-nNameServer service address, format ip:port
-bBroker address, indicating the broker where topic is located, supports only a single broker with ip:port address
-pSpecify read and write permissions for the new top (W=2|R=4|WR=6)
-cCluster name indicating the cluster topic is in (the cluster can be queried by the cluster list), -b takes precedence, and if no-b, commands are executed on all brokers in the cluster
updateOrderConfCreate, delete, and obtain kv configurations for specific namespaces from NameServer that are not currently enabled-hPrint Help
-nNameServer service address, format ip:port
-ttopic, key
-vorderConf, value
-mmethod, optional get, put, delete
allocateMQCalculate the load result of the consumer list load message queue with the average load algorithm-ttopic name
-hPrint Help
-nNameServer service address, format ip:port
-iipList, comma-separated, calculates the message queues for these IPS to load Topic
statsAllPrint Topic Subscription Relationship, TPS, Accumulation, 24h Read and Write Total, etc.-hPrint Help
-nNameServer service address, format ip:port
-aWhether to print only active topic s
-tSpecify topic

##### 2) Cluster-related

NameMeaningCommand OptionsExplain
clusterListView cluster information, cluster, BrokerName, BrokerId, TPS, etc.-mPrint more information (add the following information to print #InTotalYest, #OutTotalYest, #InTotalToday, #OutTotalToday)
-hPrint Help
-nNameServer service address, format ip:port
-iPrint interval, in seconds
clusterRTSend a message to detect each Broker RT in the cluster. Messages are sent to ${BrokerName} Topic.-aAmount, total number per probe, RT = total time / amount
-sMessage size in B
-cDetect which cluster
-pWhether to print formatted logs, split in |, not printed by default
-hPrint Help
-mOwning Room, Printing Use
-iSend interval, in seconds
-nNameServer service address, format ip:port

##### 3) Broker correlation

NameMeaningCommand OptionsExplain
updateBrokerConfigUpdating the Broker configuration file will modify the Broker.conf-bBroker address in ip:port format
-ccluster name
-kkey value
-vValue value
-hPrint Help
-nNameServer service address, format ip:port
brokerStatusView Broker statistics, status (almost all of the information you want is inside)-bBroker address, ip:port
-hPrint Help
-nNameServer service address, format ip:port
brokerConsumeStatsConsume Offset, Broker Offset, Diff, TImestamp and other information returned by Message Queue dimension for individual consumers in Broker-bBroker address, ip:port
-tRequest Timeout
-ldiff threshold beyond which to print
-oIs it a sequential topic, typically false
-hPrint Help
-nNameServer service address, format ip:port
getBrokerConfigGet Broker Configuration-bBroker address, ip:port
-nNameServer service address, format ip:port
wipeWritePermClear Broker write permissions from NameServer-bBroker address, ip:port
-nNameServer service address, format ip:port
-hPrint Help
cleanExpiredCQClean up expired Consume Queue s on Broker, which may result in expired queues if you manually reduce the number of queues-nNameServer service address, format ip:port
-hPrint Help
-bBroker address, ip:port
-cCluster name
cleanUnusedTopicClean up unused Topics on Broker, release Topic's Coume Queue from memory, and manually delete Topics will result in unused Topics-nNameServer service address, format ip:port
-hPrint Help
-bBroker address, ip:port
-cCluster name
sendMsgStatusSend message to Broker, return send status and RT-nNameServer service address, format ip:port
-hPrint Help
-bBrokerName, note that it is different from the Broker address
-sMessage size in B
-cNumber of Sends

##### 4) Message correlation

NameMeaningCommand OptionsExplain
queryMsgByIdQuery msg based on offsetMsgId. If you are using an open source console, you should use offsetMsgId. This command has other parameters. Read QueryMsgByIdSubCommand for details.-imsgId
-hPrint Help
-nNameServer service address, format ip:port
queryMsgByKeyQuery messages based on message Key-kmsgKey
-tTopic Name
-hPrint Help
-nNameServer service address, format ip:port
queryMsgByOffsetQuery messages based on Offset-bBroker name. (Note here that the name of the broker, not the address of the broker, can be found in the clusterList)
-iquery queue id
-ooffset value
-ttopic name
-hPrint Help
-nNameServer service address, format ip:port
queryMsgByUniqueKeyAccording to the msgId query, msgId is different from offsetMsgId, the differences are detailed in common operation and maintenance problems. - g, -d is used in conjunction with, after finding a message, trying to get a specific consumer to consume the message and return the result-hPrint Help
-nNameServer service address, format ip:port
-iuniqe msg id
-gconsumerGroup
-dclientId
-ttopic name
checkMsgSendRTDetects RT that sends messages to top, similar to clusterRT-hPrint Help
-nNameServer service address, format ip:port
-ttopic name
-aNumber of probes
-sMessage Size
sendMessageSend a message, either to a specific Message Queue or to a normal one, depending on the configuration.-hPrint Help
-nNameServer service address, format ip:port
-ttopic name
-pBody, message body
-kkeys
-ctags
-bBrokerName
-iqueueId
consumeMessageConsumer news. Depending on offset, start-end timest amp, and message queue consumption messages, different execution of different consumption logic can be configured, as detailed in ConsumeMessageCommand.-hPrint Help
-nNameServer service address, format ip:port
-ttopic name
-bBrokerName
-oStart spending from offset
-iqueueId
-gConsumer Grouping
-sStart timestamp in -h format
-dEnd timestamp
-cHow many messages are consumed
printMsgConsume and print messages from Broker, optional time period-hPrint Help
-nNameServer service address, format ip:port
-ttopic name
-cCharacter sets, such as UTF-8
-ssubExpress, filter expression
-bStart timestamp, format see -h
-eEnd timestamp
-dWhether to print the message body
printMsgByQueueSimilar to printMsg, but specifies Message Queue-hPrint Help
-nNameServer service address, format ip:port
-ttopic name
-iqueueId
-aBrokerName
-cCharacter sets, such as UTF-8
-ssubExpress, filter expression
-bStart timestamp, format see -h
-eEnd timestamp
-pWhether to print the message
-dWhether to print the message body
-fIs the number of tag s counted and printed
resetOffsetByTimeReset offset by timestamp, Broker and consumer will reset-hPrint Help
-nNameServer service address, format ip:port
-gConsumer Grouping
-ttopic name
-sReset offset corresponding to this timestamp
-fWhether to force reset, if false, only backtracking offset is supported, if true, regardless of the time stamp relationship between offset and consumeOffset
-cWhether to reset c++ client offset

5) Consumers, consumer groups are related

NameMeaningCommand OptionsExplain
consumerProgressView subscription group consumption to see how much messages accumulate for a specific client IP-gGroup Name of Consumer
-sWhether to print client IP
-hPrint Help
-nNameServer service address, format ip:port
consumerStatusView consumer status, including whether all subscriptions in the same group are identical, analyze whether Process Queue is stacked, return consumer jstack results, more content, user see ConsumerStatusSubCommand-hPrint Help
-nNameServer service address, format ip:port
-gconsumer group
-iclientId
-sExecute jstack or not
getConsumerStatusGet Consumer Spending Progress-gGroup Name of Consumer
-tQuery Subject
-iConsumer Client ip
-nNameServer service address, format ip:port
-hPrint Help
updateSubGroupUpdate or create a subscription relationship-nNameServer service address, format ip:port
-hPrint Help
-bBroker address
-cCluster name
-gConsumer Group Name
-sIs grouping allowed to consume
-mWhether to start with the smallest offset
-dIs it broadcast mode
-qNumber of retry queues
-rmax retries
-iWhen slaveReadEnable is open and effective, and has not yet reached the recommended BrokerId consumption from slave consumption, you can configure standby id to actively consume from standby
-wIf Broker suggests consuming from slave, configure to decide which slave to consume from, configure BrokerId, for example, 1
-aWhether to notify other consumers of load balancing when the number of consumers changes
deleteSubGroupDelete subscription relationship from Broker-nNameServer service address, format ip:port
-hPrint Help
-bBroker address
-cCluster name
-gConsumer Group Name
cloneGroupOffsetUse offset of source group in target group-nNameServer service address, format ip:port
-hPrint Help
-sSource Consumer Group
-dTarget consumer groups
-ttopic name
-oNot used yet

6) Connection correlation

NameMeaningCommand OptionsExplain
consumerConnec tionQuery Consumer's Network Connections-gGroup Name of Consumer
-nNameServer service address, format ip:port
-hPrint Help
producerConnec tionQuery Producer's network connection-gGroup Name of Producer
-tTheme Name
-nNameServer service address, format ip:port
-hPrint Help

7) NameServer correlation

NameMeaningCommand OptionsExplain
updateKvConfigUpdate the kv configuration of NameServer, which is not currently in use-sNamespace
-kkey
-vvalue
-nNameServer service address, format ip:port
-hPrint Help
deleteKvConfigDelete kv configuration for NameServer-sNamespace
-kkey
-nNameServer service address, format ip:port
-hPrint Help
getNamesrvConfigGet NameServer Configuration-nNameServer service address, format ip:port
-hPrint Help
updateNamesrvConfigModify NameServer configuration-nNameServer service address, format ip:port
-hPrint Help
-kkey
-vvalue

8) Other

NameMeaningCommand OptionsExplain
startMonitoringStart monitoring process, monitor message deletion, retry queue message count, etc.-nNameServer service address, format ip:port
-hPrint Help

4.3 Notes

  • Almost all commands require configuration - n for NameServer address in ip:port
  • Almost all commands can get help through -h
  • If there are both Broker address (-b) and Cluster Name (-c) configurations, the command will be executed with the Broker address first. If the broker address is not configured, commands are executed on all hosts in the cluster

 

 

Topics: RocketMQ