This paper mainly introduces how to upgrade RocketMQ cluster from master-slave synchronization to master-slave switching.
Firstly, the core configuration attributes related to DLedger multi-replica, i.e. RocketMQ master-slave switching, are introduced. Then, an example of master-slave synchronization cluster is attempted to be built, and then the master-slave switching function is simply tested.
1. RocketMQ DLedger multi-copy, i.e. master-slave switching core configuration parameters
Its main configuration parameters are as follows:
- enableDLegerCommitLog Whether DLedger is enabled, that is, whether RocketMQ master-slave switching is enabled, the default value is false. If master-slave switching needs to be turned on, the value needs to be set to true.
- dLegerGroup The raft group to which the node belongs is recommended to be consistent with brokerName, such as broker-a.
- dLegerPeers Cluster node information, example configuration as follows: n0-127.0.0.1:40911;n1-127.0.0.1:40912;n2-127.0.0.1:40913, multiple nodes are separated by colons in English, single entry follows the legerSlefId-ip: port, where the port is used as internal communication of the dger.
- dLegerSelfId The current node id. It's taken from the beginning of the entry in legerPeers, which is n0 in the above example, and it's especially important to emphasize that only the first character is in English and the other characters need to be configured as numbers.
- storePathRootDir The storage root directory of DLedger log files is set to a different directory from storePathCommitLog in order to support smooth upgrades.
2. Building Master-slave Synchronization Environment
Firstly, we build a master-slave synchronization architecture in the traditional sense, pour a certain amount of data into the cluster, and then upgrade to DLedger cluster.
Building a rocketmq master-slave synchronization cluster on the Linux server is not a difficult thing, I think, so this article will not elaborate on the process, only post the relevant configuration.
The deployment structure of the experimental environment is one-time, and the deployment diagram is as follows:
Next, I will focus on the broker configuration file. The broker configuration file on 220 is as follows:
brokerClusterName = DefaultCluster brokerName = broker-a brokerId = 0 deleteWhen = 04 fileReservedTime = 48 brokerRole = ASYNC_MASTER flushDiskType = ASYNC_FLUSH brokerIP1=192.168.0.220 brokerIP2=192.168.0.220 namesrvAddr=192.168.0.221:9876;192.168.0.220:9876 storePathRootDir=/opt/application/rocketmq-all-4.5.2-bin-release/store storePathCommitLog=/opt/application/rocketmq-all-4.5.2-bin-release/store/commitlog autoCreateTopicEnable=false autoCreateSubscriptionGroup=false
The broker configuration file on 221 is as follows:
brokerClusterName = DefaultCluster brokerName = broker-a brokerId = 1 deleteWhen = 04 fileReservedTime = 48 brokerRole = SLAVE flushDiskType = ASYNC_FLUSH brokerIP1=192.168.0.221 brokerIP2=192.168.0.221 namesrvAddr=192.168.0.221:9876;192.168.0.220:9876 storePathRootDir=/opt/application/rocketmq-all-4.5.2-bin-release/store storePathCommitLog=/opt/application/rocketmq-all-4.5.2-bin-release/store/commitlog autoCreateTopicEnable=false autoCreateSubscriptionGroup=false
The relevant startup commands are as follows:
nohup bin/mqnamesrv /dev/null 2>&1 & nohup bin/mqbroker -c conf/broker.conf /dev/null 2>&1 &
The cluster information after installation is shown as follows:
3. Master-slave synchronous cluster upgrade to DLedger
3.1 Deployment Architecture
DLedger cluster requires at least three machines, so building DLedger requires introducing another machine. Its deployment structure is as follows:
From master-slave synchronous cluster to DLedger cluster, users are most concerned about whether the upgraded cluster can be compatible with the original data, that is, whether the original stored messages can be consumed by message consumers or even queried. To facilitate subsequent validation, I first added a query-friendly message (setting the key of the message) to the mq cluster using the following program.
public class Producer { public static void main(String[] args) throws MQClientException, InterruptedException { DefaultMQProducer producer = new DefaultMQProducer("producer_dw_test"); producer.setNamesrvAddr("192.168.0.220:9876;192.168.0.221:9876"); producer.start(); for(int i =600000; i < 600100; i ++) { try { Message msg = new Message("topic_dw_test_by_order_01",null , "m" + i,("Hello RocketMQ" + i ).getBytes(RemotingHelper.DEFAULT_CHARSET)); SendResult sendResult = producer.send(msg); //System.out.printf("%s%n", sendResult); } catch (Exception e) { e.printStackTrace(); Thread.sleep(1000); } } producer.shutdown(); System.out.println("end"); } }
Examples of query results for messages are as follows:
3.2 Upgrade steps
Step1: Copy the rocketmq of 192.168.0.220 to 192.168.0.222, using the following commands. At 192.168.0.220, type the following command:
scp -r rocketmq-all-4.5.2-bin-release/ root@192.168.0.222:/opt/application/rocketmq-all-4.5.2-bin-release
> Warm Tip: In the example, because the version is the same, in the actual process, the version needs to be upgraded, so you need to download the latest version first, and then copy the store directory of the old cluster to the store directory of the new cluster.
Step2: Add configuration attributes related to the dledger to the broker.conf configuration files of the three servers in turn.
192.168.0.220 broker configuration file is as follows:
brokerClusterName = DefaultCluster brokerId = 0 deleteWhen = 04 fileReservedTime = 48 brokerRole = ASYNC_MASTER flushDiskType = ASYNC_FLUSH brokerIP1=192.168.0.220 brokerIP2=192.168.0.220 namesrvAddr=192.168.0.221:9876;192.168.0.220:9876 storePathRootDir=/opt/application/rocketmq-all-4.5.2-bin-release/store storePathCommitLog=/opt/application/rocketmq-all-4.5.2-bin-release/store/commitlog autoCreateTopicEnable=false autoCreateSubscriptionGroup=false # Attributes related to dledger enableDLegerCommitLog=true storePathRootDir=/opt/application/rocketmq-all-4.5.2-bin-release/store/dledger_store dLegerGroup=broker-a dLegerPeers=n0-192.168.0.220:40911;n1-192.168.0.221:40911;n2-192.168.0.222:40911 dLegerSelfId=n0
192.168.0.221 broker configuration file is as follows:
brokerClusterName = DefaultCluster brokerName = broker-a brokerId = 1 deleteWhen = 04 fileReservedTime = 48 brokerRole = SLAVE flushDiskType = ASYNC_FLUSH brokerIP1=192.168.0.221 brokerIP2=192.168.0.221 namesrvAddr=192.168.0.221:9876;192.168.0.220:9876 storePathRootDir=/opt/application/rocketmq-all-4.5.2-bin-release/store storePathCommitLog=/opt/application/rocketmq-all-4.5.2-bin-release/store/commitlog autoCreateTopicEnable=false autoCreateSubscriptionGroup=false # Configuration attributes related to dledger enableDLegerCommitLog=true storePathRootDir=/opt/application/rocketmq-all-4.5.2-bin-release/store/dledger_store dLegerGroup=broker-a dLegerPeers=n0-192.168.0.220:40911;n1-192.168.0.221:40911;n2-192.168.0.222:40911 dLegerSelfId=n1
192.168.0.222 broker configuration file is as follows:
brokerClusterName = DefaultCluster brokerName = broker-a brokerId = 0 deleteWhen = 04 fileReservedTime = 48 brokerRole = ASYNC_MASTER flushDiskType = ASYNC_FLUSH brokerIP1=192.168.0.222 brokerIP2=192.168.0.222 namesrvAddr=192.168.0.221:9876;192.168.0.220:9876 storePathRootDir=/opt/application/rocketmq-all-4.5.2-bin-release/store storePathCommitLog=/opt/application/rocketmq-all-4.5.2-bin-release/store/commitlog autoCreateTopicEnable=false autoCreateSubscriptionGroup=false # Configuration related to dledger enableDLegerCommitLog=true storePathRootDir=/opt/application/rocketmq-all-4.5.2-bin-release/store/dledger_store dLegerGroup=broker-a dLegerPeers=n0-192.168.0.220:40911;n1-192.168.0.221:40911;n2-192.168.0.222:40911 dLegerSelfId=n2
> Warm hint: legerSelfId is n0, n1, n2, respectively. In the real production environment, the storePathRootDir and storePathCommitLog in the broker configuration file try to use a separate root directory, so that their disk utilization will not affect each other.
Step3: Copy all files under store/config to the congfig directory of the dledger store.
cd /opt/application/rocketmq-all-4.5.2-bin-release/store/ cp config/* dledger_store/config/
> Warm Tip: This step can be replicated in accordance with the directory that they configure on time.
Step4: Start three broker s in turn.
nohup bin/mqbroker -c conf/broker.conf /dev/null 2>&1 &
If the startup is successful, the cluster information seen in rocketmq-console is as follows:
3.3 Verify Message Sending and Message Finding
First of all, we verify whether the message before upgrading can be queried. Then we still look for the message whose key is m600000. The result is shown in the figure.
Then let's test message delivery. The test code is as follows:
public class Producer { public static void main(String[] args) throws MQClientException, InterruptedException { DefaultMQProducer producer = new DefaultMQProducer("producer_dw_test"); producer.setNamesrvAddr("192.168.0.220:9876;192.168.0.221:9876"); producer.start(); for(int i =600200; i < 600300; i ++) { try { Message msg = new Message("topic_dw_test_by_order_01",null , "m" + i,("Hello RocketMQ" + i ).getBytes(RemotingHelper.DEFAULT_CHARSET)); SendResult sendResult = producer.send(msg); System.out.printf("%s%n", sendResult); } catch (Exception e) { e.printStackTrace(); Thread.sleep(1000); } } producer.shutdown(); System.out.println("end"); } }
The results are as follows:
Then go to the console to query the message, and the result shows that the new message can also be queried.
Finally, let's verify that the primary node is down and whether the message delivery will be affected.
In the process of message sending, to close the primary node, the screenshot is as follows:
Look again at the state of the cluster:
After the replication group completes the primary server election again, it can continue processing message sending.
> Warm Tip: Because this example is one master-one-slave, messages are not available during the election, but in the real production environment, its deployment architecture is multi-master-slave, that is, a replication group during the leader election, other replication groups can complete the sending of messages on behalf of the replication group to achieve high availability of message services.
Logs associated with DLedger are stored by default in the broker_default.log file.
This is the end of this article. If you think this article is helpful to you, I hope you can give me some compliments. Thank you.
Recommended reading: Source code analysis RocketMQ DLedger multiple copies, namely master-slave switching series articles:
1,RocketMQ Multi-copy Preface: A Preliminary Study of raft Protocol
2,Leader Selector of RocketMQ Multiple Copies for Source Code Analysis
3,Implementation of RocketMQ DLedger Multi-copy Storage for Source Code Analysis
4,Log Addition Process for Source Analysis RocketMQ DLedger
5,Log Replication of RocketMQ DLedger (Multi-copy) - Part I
6,Log Replication of RocketMQ DLedger (Multiple Copies) for Source Code Analysis
7,Design Principle of RocketMQ DLedger Multi-copy Log Replication Based on raft Protocol
8,RocketMQ integrates DLedger (multi-copy), i.e. master-slave switching to achieve smooth upgrade
9,Source Code Analysis RocketMQ DLedger Multi-copy Principle of Master-Slave Switching
10,RocketMQ smooth upgrade to master-slave switch (actual combat)
> Author's Profile: The author of RocketMQ Technology Insider, RocketMQ Community Evangelist, Maintaining Public Number: Middleware Interest Circle, can scan the following two-dimensional code to interact with the author.
