Use docker to learn the core configuration - command - principle of redis master-slave replication 3 redis sentinel "sentinel mode"

Posted by kraadde on Sun, 09 Jan 2022 11:15:11 +0100

Core configuration

More configurations in

  1. sentinel can monitor multiple master nodes and distinguish them by naming, such as master1, master2

     sentinel monitor master1 2
     sentinel monitor master2 ..
  2. Standard for sentinels to judge the downline of master nodes

     #Sentinel will ping each node. If there is still no reply after 30 seconds, it will judge whether to go offline
     sentinel down-after-milliseconds mymaster 30000
  3. What are subjective offline and objective offline
    One sentinel thinks that the master node is offline, which is called "subjective". When n stations think it is offline, it is called "objective". N is the number 2 on the last side of the first point configuration. When offline, failover is enabled.

  4. Limit the number of replication initiated to the new node after failover
    The sentinel leader does failover work. It will send a command to each slave node to initiate replication to the new master node. This parameter determines whether this step is concurrent or serial.

    If the master node has too many slave nodes, all slave nodes replicate concurrently, which will put pressure on the master node.

     #`1 ` means that only one can be copied at a time
     sentinel parallel-syncs mymaster 1
  5. Configure the password to connect to the monitored master node

     sentinel auth-pass <master-name> <password>

sentinel command

Connect any sentinel (no need to enter the container). 22531 is the external mapping port of sentinel 1. 22531 = > 26379

[root@VM-0-8-centos ~]# redis-cli -p 22531>

1 displays all monitored masters and their status

SENTINEL masters> SENTINEL masters
1)  1) "name"
    2) "mymaster" #Custom name
    3) "ip"
    4) ""
    5) "port"
    6) "6379"
  1. Displays the information and status of the specified master

     SENTINEL master <master name>> SENTINEL master mymaster
      1) "name"
      2) "mymaster"
     3) ...
  2. Displays all slave nodes of the specified master

     SENTINEL slaves <master name>> SENTINEL slaves mymaster
    1)  1) "name"
         2) ""
         3) "ip"
         4) ""
         5) "port"
         6) "6379"
    2)  1) "name"
         2) ""
         3) "ip"
         4) ""
  3. Returns the ip and port of the specified master., If a failover is in progress or has been completed, the ip and port of the slave promoted to master will be displayed.

     SENTINEL get-master-addr-by-name <master name>> SENTINEL get-master-addr-by-name mymaster
     1) ""
     2) "6379"
  4. The sentinel is forced to perform failover, and the consent of other sentinels is not required. However, the latest configuration will be sent to other sentinels after failover.

     SENTINEL failover <master name>

    Tip: if the current cli is a sentinel1 node, the sentinel1 node will perform failover.> sentinel failover mymaster

    sentinel1 log

     //Command received
     # Executing user requested FAILOVER of 'mymaster'
     //Increment the cluster status version number, which will be adopted by the new master selected next.
     # +new-epoch 4
     //Start failover of Redis cluster named "mymaster"
     # +try-failover master mymaster 6379
     //Vote for yourself as a leader
     # +vote-for-leader b0d33c8c68f57a0e3e628bf5cded4419a492177c 4
     //The selection of leaders is complete (i.e. myself)
     # +selected-leader master mymaster 6379
     //Start looking for a suitable slave in the cluster
     # +failover-state-select-slave master mymaster 6379
     // selected
     # +selected-slave slave 6379 @ mymaster 6379
     //Send the slaveof no one command to to become the master node
     * +failover-state-send-slaveof-noone slave 6379 @ mymaster 6379
     //Wait for promotion to the master node (wait for other sentinel to confirm the slave)
     * +failover-state-wait-promotion slave 6379 @ mymaster 6379
     //Promotion success (it means that all other sentinel are confirmed to be successful)
     # +promoted-slave slave 6379 @ mymaster 6379
     //Update the configuration information of all slave in the master node
     # +failover-state-reconf-slaves master mymaster 6379
     //Send the "slave of" command to the specified slave to follow the new master
     * +slave-reconf-sent slave 6379 @ mymaster 6379
     //The target slave is performing a slaveof operation
     * +slave-reconf-inprog slave 6379 @ mymaster 6379
     //After the target slave configuration information is updated, the leader can start reconfig for the next slave
     * +slave-reconf-done slave 6379 @ mymaster 6379
     //Update configuration completed
     # +failover-end master mymaster 6379
     //This failover is complete.
     //Each sentinel starts to monitor the new master
     # +switch-master mymaster 6379 6379
     * +slave slave 6379 @ mymaster 6379
     * +slave slave 6379 @ mymaster 6379

    sentinel2 and sentinel3 logs

     # +new-epoch 4
     # +config-update-from sentinel b0d33c8c68f57a0e3e628bf5cded4419a492177c 26379 @ mymaster 6379
     # +switch-master mymaster 6379 6379
     * +slave slave 6379 @ mymaster 6379
     * +slave slave 6379 @ mymaster 6379
     * +convert-to-slave slave 6379 @ mymaster 6379

    It can be seen that the sentinel cluster does not vote. sentinel1 directly completes the failover (even if the current primary node is in normal state), and then sends the latest configuration to other sentinels.

Summarize the principle of sentinel

The implementation principle of Sentinel is mainly divided into the following three steps.

  1. Problem detection mainly focuses on three scheduled tasks. These three internal execution tasks can ensure that Sentinel will know immediately if there is a problem.

  2. Finding problems mainly refers to subjective offline and objective offline. When a Sentinel machine finds a problem, it will go offline subjectively, but when multiple sentinels find a problem, it will go offline objectively.

  3. Finding the person to solve the problem mainly focuses on the leader election. How to elect a leader at multiple nodes in Sentinel.

  4. To solve the problem, we mainly talk about failover, that is, how to failover.

Three timed tasks within Sentinel.

Each Sentinel performs Info Replication on the Master and Slave every 10 seconds.

Every 2 seconds, each Sentinel exchanges information (pub/sub) through the channel of the Master node.

ping other sentinels and Redis every 1 second.

The first scheduled task means that Redis Sentinel can make failure judgment and failover for Redis nodes. There are three scheduled tasks within Redis as the basis to find Slave nodes from Info Replication. This command can determine the master-Slave relationship.

The second scheduled task is similar to publishing and subscribing. Sentinel will determine the master-slave relationship and interact through the 'sentinel:hello' channel. Understanding the master-slave relationship can help better automate Redis operation. Then sentinel will tell the system to send messages to other sentinel nodes, and finally reach a consensus. At the same time, sentinel nodes can perceive each other.

The third timing task refers to heartbeat detection for each node and other Sentinel, which is the basis for failure determination.

How sentinel selects the appropriate Slave node

Redis actually has a priority configuration. In the configuration file, the parameter slave priority is the priority configuration of the sell node. If it exists, it returns. If it does not exist, it continues.

When the above priority is not satisfied, Redis will also select the Slave node with the largest copy offset. If it exists, it will return. If it does not exist, it will continue. The reason for choosing the largest offset is that the smaller the offset, the less close it is to the Master data. Now the Master hangs up, indicating that there may be problems with the machine data with a small offset. This is why the Slave with the largest offset is selected.

If the offsets are the same, Redis will select the node with the smallest runid by default.

Topics: server