1 Introduction to roles
- Producer: The sender of the message; Example: Sender
- Consumer: Message recipient; Example: Recipient
- Broker: Stay and transfer messages; Example: Post Office
- NameServer: Manage Broker; Examples: the administrative bodies of various post offices
- Topic: Differentiate the types of messages; One sender can send a message to one or more Topics; Recipients of a message can subscribe to one or more Topic messages
- Message Queue: Equivalent to a partition of Topic; For sending and receiving messages in parallel
2 Cluster Setup
2.1 Cluster Characteristics
-
NameServer is a virtually stateless node that can be deployed in a cluster without any information synchronization between nodes.
-
Broker deployment is relatively complex. Broker is divided into Master and Slave. A Master can correspond to multiple Slaves, but a Slave can only correspond to one Master. The corresponding relationship between Master and Slave is defined by specifying the same BrokerName, different BrokerId s, 0 for Master and non-0 for Slave. Master can also deploy multiple. Each broker establishes a long connection to all nodes in the NameServer cluster and periodically registers Topic information to all NameServers.
-
Producer establishes a long connection with one of the nodes in the NameServer cluster (randomly selected), periodically fetches Topic routing information from NameServer, establishes a long connection to the Master providing Topic services, and periodically sends a heartbeat to the Master. Producer is completely stateless and can be deployed in clusters.
-
Consumer establishes a long connection with one of the nodes in the NameServer cluster (randomly selected), periodically fetches Topic routing information from NameServer, establishes a long connection to Master and Slave providing Topic services, and periodically sends heartbeats to Master and Slave. Consumers can subscribe to messages from either Master or Slave, and the subscription rules are determined by the Broker configuration.
2.2 Cluster Mode
1) Single Master Mode
This is a risky approach that will make the entire service unavailable once the Broker restarts or goes down. Online environments are not recommended and can be used for local testing.
2) Multiple Master Mode
A cluster has no Slave and all Masters, such as 2 Masters or 3 Masters. The advantages and disadvantages of this pattern are as follows:
- Advantages: Simple configuration, single Master downtime or restart maintenance has no effect on the application. When the disk is configured as RAID10, the message will not be lost even if the machine is not restored when the disk is configured as RAID10, because RAID10 disk is very reliable (asynchronous brush disc loses a few messages, synchronous brush disc does not lose one), and the performance is the highest;
- Disadvantages: During downtime of a single machine, messages that are not consumed on this machine cannot be subscribed until the machine is restored, and message real-time will be affected.
3) Multiple Master Multiple Slave Mode (Asynchronous)
Each Master configures a Slave with multiple pairs of Master-Slave s, HA replicates asynchronously, and primary has a short message delay (in milliseconds). The advantages and disadvantages of this mode are as follows:
- Advantages: Even if the disk is damaged, messages are lost very little, and message real-time is not affected, and consumers can still consume from Slave after Master is down, and this process is transparent to applications, does not require manual intervention, and performs almost the same as in multiple Master mode;
- Disadvantages: Master downtime, small amounts of messages will be lost if the disk is damaged.
4) Multiple Master Multiple Slave Mode (Synchronization)
Each Master is configured with a Slave, with multiple pairs of Master-Slave s. HA uses synchronous dual writing, that is, only if the primary and standby are successfully written, will it return success to the application. The advantages and disadvantages of this mode are as follows:
- Advantages: There is no single point of failure for both data and services. In case of Master downtime, there is no delay for messages, and service availability and data availability are very high.
- Disadvantages: performance is slightly lower than asynchronous replication mode (about 10 percent lower), RT to send a single message is slightly higher, and the standby cannot automatically switch to the host after the primary node is down.
3 Double Primary and Double Slave Clusters
3.1 Overall Architecture
Message High Availability with 2master-2slave
3.2 Cluster Workflow
- Start NameServer, and when NameServer gets up, listen on the port and wait for Broker, Producer, and Consumer to connect, which is equivalent to a routing control center.
- The broker starts, maintains a long connection to all NameServers, and periodically sends a heartbeat packet. The heartbeat package contains the current Broker information (IP+port, etc.) and stores all Topic information. After successful registration, Topic maps to Broker in the NameServer cluster.
- Before sending and receiving a message, create a Topic, specify which brokers the Topic will store, or automatically create a Topic when sending a message.
- Producer sends a message, makes a long connection to one of the NameServer clusters at startup, obtains from NameServer which brokers are currently sent on Topic, polls to select a queue from the queue list, and then makes a long connection to the Broker where the queue is located to send messages to the Broker.
- Consumer is similar to Producer in that it establishes a long connection with one of the NameServer s, gets which brokers are on the Topic currently subscribed to, and then establishes a connection channel directly with the Broker to start consuming messages.
3.3 Server Environment
Sequence Number | IP | role | Architecture mode |
---|---|---|---|
1 | 192.168.2.12 | nameserver,brokerserver | Master1,Slave2 |
2 | 192.168.2.13 | nameserver,brokerserver | Master2,Slave1 |
3.4 Host Add Information
vim /etc/hosts
The configuration is as follows:
# nameserver 192.168.2.12 rocketmq-nameserver1 192.168.2.13 rocketmq-nameserver2 # broker 192.168.2.12 rocketmq-master1 192.168.2.12 rocketmq-slave2 192.168.2.13 rocketmq-master2 192.168.2.13 rocketmq-slave1
Restart network card after configuration is complete
systemctl restart network
3.5 Firewall Configuration
Hosts need remote access to the virtual machine's rocketmq services and web services, open related port numbers, and simply, roughly, close the firewall directly
bash
# Close Firewall systemctl stop firewalld.service # View the status of your firewall firewall-cmd --state # Disable firewall startup systemctl disable firewalld.service
Or, for security reasons, only certain port numbers are open, RocketMQ uses three ports by default: 9876, 10911, 11011. If the firewall is not closed, then the firewall must open these ports:
- nameserver uses port 9876 by default
- master uses port 10911 by default
- slave uses port 11011 by default
Execute the following commands:
# Open name server default port firewall-cmd --remove-port=9876/tcp --permanent # Open master default port firewall-cmd --remove-port=10911/tcp --permanent # Open slave default port (current cluster mode is not open) firewall-cmd --remove-port=11011/tcp --permanent # service iptables restart firewall-cmd --reload
3.6 Configuration of environment variables
vim /etc/profile
Add the following command at the end of the profile file
#set rocketmq This directory is the rocketmq binary package decompression directory ROCKETMQ_HOME=/usr/local/rocketmq PATH=$PATH:$ROCKETMQ_HOME/bin export ROCKETMQ_HOME PATH
Input: wq! Save and exit, and make the configuration take effect immediately:
bash
source /etc/profile
3.7 Create message store path
mkdir /usr/local/rocketmq/store mkdir /usr/local/rocketmq/store/commitlog mkdir /usr/local/rocketmq/store/consumequeue mkdir /usr/local/rocketmq/store/index
3.8 broker Profile
1)master1
Server: 192.168.2.12
vim /usr/local/rocketmq/conf/2m-2s-sync/new-broker-a.properties
Modify the configuration as follows:
#The name of the cluster to which you belong brokerClusterName=rocketmq-cluster #broker name, note that different profiles here fill in different brokerName=broker-a #0 for Master, >0 for Slave brokerId=0 #Name Server address, semicolon-separated namesrvAddr=rocketmq-nameserver1:9876;rocketmq-nameserver2:9876 #When sending a message, automatically create a topic that does not exist on the server, default number of queues created defaultTopicQueueNums=4 #Whether Broker is allowed to automatically create Topic, it is recommended to turn it on offline and turn it off online autoCreateTopicEnable=true #Whether Broker is allowed to automatically create subscription groups, it is recommended to open offline and close Online autoCreateSubscriptionGroup=true #Broker's listening port for external services listenPort=10911 #Delete file point in time, default 4 am deleteWhen=04 #File retention time, default 48 hours fileReservedTime=120 #commitLog default size of 1G per file mapedFileSizeCommitLog=1073741824 #ConsumeQueue saves 30W bars per file by default, adjusting to business conditions mapedFileSizeConsumeQueue=300000 #destroyMapedFileIntervalForcibly=120000 #redeleteHangedFileInterval=120000 #Detect physical file disk space diskMaxUsedSpaceRatio=88 #Storage Path storePathRootDir=/usr/local/rocketmq/store #commitLog storage path storePathCommitLog=/usr/local/rocketmq/store/commitlog #Consumer Queue Storage Path Storage Path storePathConsumeQueue=/usr/local/rocketmq/store/consumequeue #Message Index Storage Path storePathIndex=/usr/local/rocketmq/store/index #checkpoint file storage path storeCheckpoint=/usr/local/rocketmq/store/checkpoint #abort file storage path abortFile=/usr/local/rocketmq/store/abort #Limited message size maxMessageSize=65536 #flushCommitLogLeastPages=4 #flushConsumeQueueLeastPages=2 #flushCommitLogThoroughInterval=10000 #flushConsumeQueueThoroughInterval=60000 #Broker's role #- ASYNC_MASTER Asynchronous Replication Master #- SYNC_MASTER Synchronized Double Write Master #- SLAVE brokerRole=SYNC_MASTER #Brush disc mode #- ASYNC_FLUSH asynchronous brush disc #- SYNC_FLUSH synchronous brush disc flushDiskType=SYNC_FLUSH #checkTransactionMessageEnable=false #Number of messaging thread pools #sendMessageThreadPoolNums=128 #Number of pull message thread pools #pullMessageThreadPoolNums=128
2)slave2
Server: 192.168.2.12
vim /usr/local/rocketmq/conf/2m-2s-sync/new-broker-b-s.properties
Modify the configuration as follows:
#The name of the cluster to which you belong brokerClusterName=rocketmq-cluster #broker name, note that different profiles here fill in different brokerName=broker-b #0 for Master, >0 for Slave brokerId=1 #Name Server address, semicolon-separated namesrvAddr=rocketmq-nameserver1:9876;rocketmq-nameserver2:9876 #When sending a message, automatically create a topic that does not exist on the server, default number of queues created defaultTopicQueueNums=4 #Whether Broker is allowed to automatically create Topic, it is recommended to turn it on offline and turn it off online autoCreateTopicEnable=true #Whether Broker is allowed to automatically create subscription groups, it is recommended to open offline and close Online autoCreateSubscriptionGroup=true #Broker's listening port for external services listenPort=11011 #Delete file point in time, default 4 am deleteWhen=04 #File retention time, default 48 hours fileReservedTime=120 #commitLog default size of 1G per file mapedFileSizeCommitLog=1073741824 #ConsumeQueue saves 30W bars per file by default, adjusting to business conditions mapedFileSizeConsumeQueue=300000 #destroyMapedFileIntervalForcibly=120000 #redeleteHangedFileInterval=120000 #Detect physical file disk space diskMaxUsedSpaceRatio=88 #Storage Path storePathRootDir=/usr/local/rocketmq/store #commitLog storage path storePathCommitLog=/usr/local/rocketmq/store/commitlog #Consumer Queue Storage Path Storage Path storePathConsumeQueue=/usr/local/rocketmq/store/consumequeue #Message Index Storage Path storePathIndex=/usr/local/rocketmq/store/index #checkpoint file storage path storeCheckpoint=/usr/local/rocketmq/store/checkpoint #abort file storage path abortFile=/usr/local/rocketmq/store/abort #Limited message size maxMessageSize=65536 #flushCommitLogLeastPages=4 #flushConsumeQueueLeastPages=2 #flushCommitLogThoroughInterval=10000 #flushConsumeQueueThoroughInterval=60000 #Broker's role #- ASYNC_MASTER Asynchronous Replication Master #- SYNC_MASTER Synchronized Double Write Master #- SLAVE brokerRole=SLAVE #Brush disc mode #- ASYNC_FLUSH asynchronous brush disc #- SYNC_FLUSH synchronous brush disc flushDiskType=ASYNC_FLUSH #checkTransactionMessageEnable=false #Number of messaging thread pools #sendMessageThreadPoolNums=128 #Number of pull message thread pools #pullMessageThreadPoolNums=128
3)master2
Server: 192.168.2.13
vim /usr/local/rocketmq/conf/2m-2s-sync/new-broker-b.properties
Modify the configuration as follows:
#The name of the cluster to which you belong brokerClusterName=rocketmq-cluster #broker name, note that different profiles here fill in different brokerName=broker-b #0 for Master, >0 for Slave brokerId=0 #Name Server address, semicolon-separated namesrvAddr=rocketmq-nameserver1:9876;rocketmq-nameserver2:9876 #When sending a message, automatically create a topic that does not exist on the server, default number of queues created defaultTopicQueueNums=4 #Whether Broker is allowed to automatically create Topic, it is recommended to turn it on offline and turn it off online autoCreateTopicEnable=true #Whether Broker is allowed to automatically create subscription groups, it is recommended to open offline and close Online autoCreateSubscriptionGroup=true #Broker's listening port for external services listenPort=10911 #Delete file point in time, default 4 am deleteWhen=04 #File retention time, default 48 hours fileReservedTime=120 #commitLog default size of 1G per file mapedFileSizeCommitLog=1073741824 #ConsumeQueue saves 30W bars per file by default, adjusting to business conditions mapedFileSizeConsumeQueue=300000 #destroyMapedFileIntervalForcibly=120000 #redeleteHangedFileInterval=120000 #Detect physical file disk space diskMaxUsedSpaceRatio=88 #Storage Path storePathRootDir=/usr/local/rocketmq/store #commitLog storage path storePathCommitLog=/usr/local/rocketmq/store/commitlog #Consumer Queue Storage Path Storage Path storePathConsumeQueue=/usr/local/rocketmq/store/consumequeue #Message Index Storage Path storePathIndex=/usr/local/rocketmq/store/index #checkpoint file storage path storeCheckpoint=/usr/local/rocketmq/store/checkpoint #abort file storage path abortFile=/usr/local/rocketmq/store/abort #Limited message size maxMessageSize=65536 #flushCommitLogLeastPages=4 #flushConsumeQueueLeastPages=2 #flushCommitLogThoroughInterval=10000 #flushConsumeQueueThoroughInterval=60000 #Broker's role #- ASYNC_MASTER Asynchronous Replication Master #- SYNC_MASTER Synchronized Double Write Master #- SLAVE brokerRole=SYNC_MASTER #Brush disc mode #- ASYNC_FLUSH asynchronous brush disc #- SYNC_FLUSH synchronous brush disc flushDiskType=SYNC_FLUSH #checkTransactionMessageEnable=false #Number of messaging thread pools #sendMessageThreadPoolNums=128 #Number of pull message thread pools #pullMessageThreadPoolNums=128
4)slave1
Server: 192.168.2.13
vim /usr/local/rocketmq/conf/2m-2s-sync/new-broker-a-s.properties
Modify the configuration as follows:
#The name of the cluster to which you belong brokerClusterName=rocketmq-cluster #broker name, note that different profiles here fill in different brokerName=broker-a #0 for Master, >0 for Slave brokerId=1 #Name Server address, semicolon-separated namesrvAddr=rocketmq-nameserver1:9876;rocketmq-nameserver2:9876 #When sending a message, automatically create a topic that does not exist on the server, default number of queues created defaultTopicQueueNums=4 #Whether Broker is allowed to automatically create Topic, it is recommended to turn it on offline and turn it off online autoCreateTopicEnable=true #Whether Broker is allowed to automatically create subscription groups, it is recommended to open offline and close Online autoCreateSubscriptionGroup=true #Broker's listening port for external services listenPort=11011 #Delete file point in time, default 4 am deleteWhen=04 #File retention time, default 48 hours fileReservedTime=120 #commitLog default size of 1G per file mapedFileSizeCommitLog=1073741824 #ConsumeQueue saves 30W bars per file by default, adjusting to business conditions mapedFileSizeConsumeQueue=300000 #destroyMapedFileIntervalForcibly=120000 #redeleteHangedFileInterval=120000 #Detect physical file disk space diskMaxUsedSpaceRatio=88 #Storage Path storePathRootDir=/usr/local/rocketmq/store #commitLog storage path storePathCommitLog=/usr/local/rocketmq/store/commitlog #Consumer Queue Storage Path Storage Path storePathConsumeQueue=/usr/local/rocketmq/store/consumequeue #Message Index Storage Path storePathIndex=/usr/local/rocketmq/store/index #checkpoint file storage path storeCheckpoint=/usr/local/rocketmq/store/checkpoint #abort file storage path abortFile=/usr/local/rocketmq/store/abort #Limited message size maxMessageSize=65536 #flushCommitLogLeastPages=4 #flushConsumeQueueLeastPages=2 #flushCommitLogThoroughInterval=10000 #flushConsumeQueueThoroughInterval=60000 #Broker's role #- ASYNC_MASTER Asynchronous Replication Master #- SYNC_MASTER Synchronized Double Write Master #- SLAVE brokerRole=SLAVE #Brush disc mode #- ASYNC_FLUSH asynchronous brush disc #- SYNC_FLUSH synchronous brush disc flushDiskType=ASYNC_FLUSH #checkTransactionMessageEnable=false #Number of messaging thread pools #sendMessageThreadPoolNums=128 #Number of pull message thread pools #pullMessageThreadPoolNums=128
3.9 Modify startup script file
1)runbroker.sh
sh
vi /usr/local/rocketmq/bin/runbroker.sh
JVM parameters need to be adjusted appropriately according to memory size:
bash
#=================================================== # Development Environment Configuration JVM Configuration JAVA_OPT="${JAVA_OPT} -server -Xms256m -Xmx256m -Xmn128m"
2)runserver.sh
vim /usr/local/rocketmq/bin/runserver.sh
JAVA_OPT="${JAVA_OPT} -server -Xms256m -Xmx256m -Xmn128m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m"
3.10 Service Startup
1) Start the NameServer cluster
NameServer was started in 192.168.2.12 and 192.168.2.13, respectively.
bash
cd /usr/local/rocketmq/bin nohup sh mqnamesrv &
2) Start Broker Cluster
- Start master 1 and slave2 on 192.168.2.12
master1:
cd /usr/local/rocketmq/bin nohup sh mqbroker -c /usr/local/rocketmq/conf/2m-2s-sync/new-broker-a.properties &
slave2:
cd /usr/local/rocketmq/bin nohup sh mqbroker -c /usr/local/rocketmq/conf/2m-2s-sync/new-broker-b-s.properties &
- Start Master 2 and slave2 on 192.168.2.13
master2
cd /usr/local/rocketmq/bin nohup sh mqbroker -c /usr/local/rocketmq/conf/2m-2s-sync/new-broker-b.properties &
slave1
cd /usr/local/rocketmq/bin nohup sh mqbroker -c /usr/local/rocketmq/conf/2m-2s-sync/new-broker-a-s.properties &
3.11 View process status
View startup process via JPS after startup
3.12 View Log
# View the nameServer log tail -500f ~/logs/rocketmqlogs/namesrv.log # View broker log tail -500f ~/logs/rocketmqlogs/broker.log
4. mqadmin management tools
4.1 Usage
Enter the RocketMQ installation location and execute in the bin directory. / mqadmin {command} {args}
4.2 Introduction to Commands
1) Topic correlation
Name | Meaning | Command Options | Explain |
updateTopic | Create Update Topic Configuration | -b | Broker address, indicating the broker where topic is located, supports only a single broker with ip:port address |
-c | Cluster name, indicating the cluster in which topic resides (clusters can be queried through a clusterList) | ||
-h- | Print Help | ||
-n | NameServer service address, format ip:port | ||
-p | Specify read and write permissions for the new top (W=2|R=4|WR=6) | ||
-r | Number of readable queues (default is 8) | ||
-w | Number of writable queues (default is 8) | ||
-t | topic name (name can only use the character ^[a-zA-Z0-9_-]+$) | ||
deleteTopic | Delete Topic | -c | Cluster name indicating the deletion of a topic under a cluster (the cluster can be queried through the cluster list) |
-h | Print Help | ||
-n | NameServer service address, format ip:port | ||
-t | topic name (name can only use the character ^[a-zA-Z0-9_-]+$) | ||
topicList | View Topic List Information | -h | Print Help |
-c | Do not configure -c to return only the topic list, increase-c to return clusterName, topic, consumerGroup information, that is, the cluster to which topic belongs and the subscription relationship, with no parameters | ||
-n | NameServer service address, format ip:port | ||
topicRoute | View Topic Routing Information | -t | topic name |
-h | Print Help | ||
-n | NameServer service address, format ip:port | ||
topicStatus | View Topic Message Queue offset | -t | topic name |
-h | Print Help | ||
-n | NameServer service address, format ip:port | ||
topicClusterList | View the list of clusters Topic is in | -t | topic name |
-h | Print Help | ||
-n | NameServer service address, format ip:port | ||
updateTopicPerm | Update Topic Read and Write Permissions | -t | topic name |
-h | Print Help | ||
-n | NameServer service address, format ip:port | ||
-b | Broker address, indicating the broker where topic is located, supports only a single broker with ip:port address | ||
-p | Specify read and write permissions for the new top (W=2|R=4|WR=6) | ||
-c | Cluster name indicating the cluster topic is in (the cluster can be queried by the cluster list), -b takes precedence, and if no-b, commands are executed on all brokers in the cluster | ||
updateOrderConf | Create, delete, and obtain kv configurations for specific namespaces from NameServer that are not currently enabled | -h | Print Help |
-n | NameServer service address, format ip:port | ||
-t | topic, key | ||
-v | orderConf, value | ||
-m | method, optional get, put, delete | ||
allocateMQ | Calculate the load result of the consumer list load message queue with the average load algorithm | -t | topic name |
-h | Print Help | ||
-n | NameServer service address, format ip:port | ||
-i | ipList, comma-separated, calculates the message queues for these IPS to load Topic | ||
statsAll | Print Topic Subscription Relationship, TPS, Accumulation, 24h Read and Write Total, etc. | -h | Print Help |
-n | NameServer service address, format ip:port | ||
-a | Whether to print only active topic s | ||
-t | Specify topic |
##### 2) Cluster-related
Name | Meaning | Command Options | Explain |
clusterList | View cluster information, cluster, BrokerName, BrokerId, TPS, etc. | -m | Print more information (add the following information to print #InTotalYest, #OutTotalYest, #InTotalToday, #OutTotalToday) |
-h | Print Help | ||
-n | NameServer service address, format ip:port | ||
-i | Print interval, in seconds | ||
clusterRT | Send a message to detect each Broker RT in the cluster. Messages are sent to ${BrokerName} Topic. | -a | Amount, total number per probe, RT = total time / amount |
-s | Message size in B | ||
-c | Detect which cluster | ||
-p | Whether to print formatted logs, split in |, not printed by default | ||
-h | Print Help | ||
-m | Owning Room, Printing Use | ||
-i | Send interval, in seconds | ||
-n | NameServer service address, format ip:port |
##### 3) Broker correlation
Name | Meaning | Command Options | Explain |
updateBrokerConfig | Updating the Broker configuration file will modify the Broker.conf | -b | Broker address in ip:port format |
-c | cluster name | ||
-k | key value | ||
-v | Value value | ||
-h | Print Help | ||
-n | NameServer service address, format ip:port | ||
brokerStatus | View Broker statistics, status (almost all of the information you want is inside) | -b | Broker address, ip:port |
-h | Print Help | ||
-n | NameServer service address, format ip:port | ||
brokerConsumeStats | Consume Offset, Broker Offset, Diff, TImestamp and other information returned by Message Queue dimension for individual consumers in Broker | -b | Broker address, ip:port |
-t | Request Timeout | ||
-l | diff threshold beyond which to print | ||
-o | Is it a sequential topic, typically false | ||
-h | Print Help | ||
-n | NameServer service address, format ip:port | ||
getBrokerConfig | Get Broker Configuration | -b | Broker address, ip:port |
-n | NameServer service address, format ip:port | ||
wipeWritePerm | Clear Broker write permissions from NameServer | -b | Broker address, ip:port |
-n | NameServer service address, format ip:port | ||
-h | Print Help | ||
cleanExpiredCQ | Clean up expired Consume Queue s on Broker, which may result in expired queues if you manually reduce the number of queues | -n | NameServer service address, format ip:port |
-h | Print Help | ||
-b | Broker address, ip:port | ||
-c | Cluster name | ||
cleanUnusedTopic | Clean up unused Topics on Broker, release Topic's Coume Queue from memory, and manually delete Topics will result in unused Topics | -n | NameServer service address, format ip:port |
-h | Print Help | ||
-b | Broker address, ip:port | ||
-c | Cluster name | ||
sendMsgStatus | Send message to Broker, return send status and RT | -n | NameServer service address, format ip:port |
-h | Print Help | ||
-b | BrokerName, note that it is different from the Broker address | ||
-s | Message size in B | ||
-c | Number of Sends |
##### 4) Message correlation
Name | Meaning | Command Options | Explain |
queryMsgById | Query msg based on offsetMsgId. If you are using an open source console, you should use offsetMsgId. This command has other parameters. Read QueryMsgByIdSubCommand for details. | -i | msgId |
-h | Print Help | ||
-n | NameServer service address, format ip:port | ||
queryMsgByKey | Query messages based on message Key | -k | msgKey |
-t | Topic Name | ||
-h | Print Help | ||
-n | NameServer service address, format ip:port | ||
queryMsgByOffset | Query messages based on Offset | -b | Broker name. (Note here that the name of the broker, not the address of the broker, can be found in the clusterList) |
-i | query queue id | ||
-o | offset value | ||
-t | topic name | ||
-h | Print Help | ||
-n | NameServer service address, format ip:port | ||
queryMsgByUniqueKey | According to the msgId query, msgId is different from offsetMsgId, the differences are detailed in common operation and maintenance problems. - g, -d is used in conjunction with, after finding a message, trying to get a specific consumer to consume the message and return the result | -h | Print Help |
-n | NameServer service address, format ip:port | ||
-i | uniqe msg id | ||
-g | consumerGroup | ||
-d | clientId | ||
-t | topic name | ||
checkMsgSendRT | Detects RT that sends messages to top, similar to clusterRT | -h | Print Help |
-n | NameServer service address, format ip:port | ||
-t | topic name | ||
-a | Number of probes | ||
-s | Message Size | ||
sendMessage | Send a message, either to a specific Message Queue or to a normal one, depending on the configuration. | -h | Print Help |
-n | NameServer service address, format ip:port | ||
-t | topic name | ||
-p | Body, message body | ||
-k | keys | ||
-c | tags | ||
-b | BrokerName | ||
-i | queueId | ||
consumeMessage | Consumer news. Depending on offset, start-end timest amp, and message queue consumption messages, different execution of different consumption logic can be configured, as detailed in ConsumeMessageCommand. | -h | Print Help |
-n | NameServer service address, format ip:port | ||
-t | topic name | ||
-b | BrokerName | ||
-o | Start spending from offset | ||
-i | queueId | ||
-g | Consumer Grouping | ||
-s | Start timestamp in -h format | ||
-d | End timestamp | ||
-c | How many messages are consumed | ||
printMsg | Consume and print messages from Broker, optional time period | -h | Print Help |
-n | NameServer service address, format ip:port | ||
-t | topic name | ||
-c | Character sets, such as UTF-8 | ||
-s | subExpress, filter expression | ||
-b | Start timestamp, format see -h | ||
-e | End timestamp | ||
-d | Whether to print the message body | ||
printMsgByQueue | Similar to printMsg, but specifies Message Queue | -h | Print Help |
-n | NameServer service address, format ip:port | ||
-t | topic name | ||
-i | queueId | ||
-a | BrokerName | ||
-c | Character sets, such as UTF-8 | ||
-s | subExpress, filter expression | ||
-b | Start timestamp, format see -h | ||
-e | End timestamp | ||
-p | Whether to print the message | ||
-d | Whether to print the message body | ||
-f | Is the number of tag s counted and printed | ||
resetOffsetByTime | Reset offset by timestamp, Broker and consumer will reset | -h | Print Help |
-n | NameServer service address, format ip:port | ||
-g | Consumer Grouping | ||
-t | topic name | ||
-s | Reset offset corresponding to this timestamp | ||
-f | Whether to force reset, if false, only backtracking offset is supported, if true, regardless of the time stamp relationship between offset and consumeOffset | ||
-c | Whether to reset c++ client offset |
5) Consumers, consumer groups are related
Name | Meaning | Command Options | Explain |
consumerProgress | View subscription group consumption to see how much messages accumulate for a specific client IP | -g | Group Name of Consumer |
-s | Whether to print client IP | ||
-h | Print Help | ||
-n | NameServer service address, format ip:port | ||
consumerStatus | View consumer status, including whether all subscriptions in the same group are identical, analyze whether Process Queue is stacked, return consumer jstack results, more content, user see ConsumerStatusSubCommand | -h | Print Help |
-n | NameServer service address, format ip:port | ||
-g | consumer group | ||
-i | clientId | ||
-s | Execute jstack or not | ||
getConsumerStatus | Get Consumer Spending Progress | -g | Group Name of Consumer |
-t | Query Subject | ||
-i | Consumer Client ip | ||
-n | NameServer service address, format ip:port | ||
-h | Print Help | ||
updateSubGroup | Update or create a subscription relationship | -n | NameServer service address, format ip:port |
-h | Print Help | ||
-b | Broker address | ||
-c | Cluster name | ||
-g | Consumer Group Name | ||
-s | Is grouping allowed to consume | ||
-m | Whether to start with the smallest offset | ||
-d | Is it broadcast mode | ||
-q | Number of retry queues | ||
-r | max retries | ||
-i | When slaveReadEnable is open and effective, and has not yet reached the recommended BrokerId consumption from slave consumption, you can configure standby id to actively consume from standby | ||
-w | If Broker suggests consuming from slave, configure to decide which slave to consume from, configure BrokerId, for example, 1 | ||
-a | Whether to notify other consumers of load balancing when the number of consumers changes | ||
deleteSubGroup | Delete subscription relationship from Broker | -n | NameServer service address, format ip:port |
-h | Print Help | ||
-b | Broker address | ||
-c | Cluster name | ||
-g | Consumer Group Name | ||
cloneGroupOffset | Use offset of source group in target group | -n | NameServer service address, format ip:port |
-h | Print Help | ||
-s | Source Consumer Group | ||
-d | Target consumer groups | ||
-t | topic name | ||
-o | Not used yet |
6) Connection correlation
Name | Meaning | Command Options | Explain |
consumerConnec tion | Query Consumer's Network Connections | -g | Group Name of Consumer |
-n | NameServer service address, format ip:port | ||
-h | Print Help | ||
producerConnec tion | Query Producer's network connection | -g | Group Name of Producer |
-t | Theme Name | ||
-n | NameServer service address, format ip:port | ||
-h | Print Help |
7) NameServer correlation
Name | Meaning | Command Options | Explain |
updateKvConfig | Update the kv configuration of NameServer, which is not currently in use | -s | Namespace |
-k | key | ||
-v | value | ||
-n | NameServer service address, format ip:port | ||
-h | Print Help | ||
deleteKvConfig | Delete kv configuration for NameServer | -s | Namespace |
-k | key | ||
-n | NameServer service address, format ip:port | ||
-h | Print Help | ||
getNamesrvConfig | Get NameServer Configuration | -n | NameServer service address, format ip:port |
-h | Print Help | ||
updateNamesrvConfig | Modify NameServer configuration | -n | NameServer service address, format ip:port |
-h | Print Help | ||
-k | key | ||
-v | value |
8) Other
Name | Meaning | Command Options | Explain |
startMonitoring | Start monitoring process, monitor message deletion, retry queue message count, etc. | -n | NameServer service address, format ip:port |
-h | Print Help |
4.3 Notes
- Almost all commands require configuration - n for NameServer address in ip:port
- Almost all commands can get help through -h
- If there are both Broker address (-b) and Cluster Name (-c) configurations, the command will be executed with the Broker address first. If the broker address is not configured, commands are executed on all hosts in the cluster