I. Introduction

• background

• The master-slave is built in front. When the master server goes down, one slave server needs to be manually switched to the master server. Manual intervention is laborious and laborious, and the service will not be available for a period of time

• Sentinel mode introduction

  • Redis provides sentry commands and is an independent process
  • The principle sentry sends commands to multiple nodes and waits for the response of the Redis server, so as to monitor the operation of multiple running Redis instances
  • When the sentinel detects that the master is down, it will automatically switch the slave to the master, and change the configuration file to switch the host by notifying other slave servers

• Sentinel's three major tasks

  • Monitoring

    • Sentinel will constantly check whether your master and slave servers are working properly

  • Reminder (Notification)

    • When a monitored Redis server has a problem, Sentinel can send a notification to the administrator or other applications through the API

  • Automatic failover

    • When a master server fails to work normally, Sentinel will start an automatic failover operation, which will upgrade one of the failed master servers from the server to the new master server, and change the other slave servers of the failed master server to replicate the new master server
    • When the client tries to connect to the failed primary server, the cluster will also return the address of the new primary server to the client, so that the cluster can use the new primary server instead of the failed server

  • problem

    • A sentinel process monitors the Redis server, which may cause problems
    • Generally, multiple sentinels are used for monitoring, and each sentinel will be monitored to form a multi sentinel mode

• Introduction to offline name of multi sentry mode

  • Subjectively Down (SDOWN for short)

    • It refers to the offline judgment made by a single Sentinel instance on the server, such as network problems and failure to receive notifications
    • If a server does not return a valid reply to the Sentinel sending the PING command within the time specified in the down after milliseconds option, Sentinel will mark the server as offline

  • Objective down (ODOWN for short)

    • It refers to the server offline judgment obtained after multiple Sentinel instances make SDOWN judgment on the same server and communicate with each other through the Sentinel is master down by addr command
    • A Sentinel can send a Sentinel is master down by addr command to another Sentinel to ask if the other Sentinel thinks the given server is offline
    • Objective offline conditions apply only to the primary server

  • Arbitration qurum

    • If Sentinel receives [enough] offline reports from other sentinels within a given time range, Sentinel will change the status of the primary server from subjective offline to objective offline
    • This [sufficient quantity] is the value in the configuration file. Generally, it is half of the number of sentinels plus 1. For example, three sentinels are set to 2
    • Down after milliseconds is a sentinel who thinks the host is unavailable after it still fails to respond after a specified time
    • When the number of sentinels who think they are offline reaches the number configured by sentinel monitor, a vote will be initiated to fail over

II. Construction

• Core process

  • ping every second. If there is no response after time, it is considered as offline
  • If more than one is satisfied, it is considered to be an objective offline
  • Vote to select master node
  • If not enough nodes agree to the master offline, the status will be removed

• Environmental preparation

  • Configure 3 sentinels, and the configuration of each sentinel is the same
  • Start sequence: start the master first, then start the slave, and finally start three sentinels
  • The sentinel port is [26379]. Remember to open it

    #Unlimited ip
    bind 0.0.0.0
    ​
    # Let sentinel service run in the background
    daemonize yes
    ​
    # Configure the master server for listening. mymaster represents the name of the server. It is user-defined. 172.18 172.109 represents the monitored master server, 6379 represents the port,
    #2 means that only when two or more sentinels think that the primary server is unavailable, the failover operation will be carried out.
    sentinel monitor mymaster 172.18.172.109 6379 2
    ​
    # Sentinel auth pass defines the password of the service. mymaster is the service name and 123456 is the Redis server password
    sentinel auth-pass mymaster 123456
    ​
    #If the master is not connected for more than 5 seconds, it is considered that the master has stopped
    sentinel down-after-milliseconds mymaster 5000
    ​
    #If the failover operation is not completed within this time, it is considered that this failover has failed
    sentinel failover-timeout mymaster 30000

  • Create three files sentinel-1 in the directory conf,sentinel-2.conf,sentinel-3.conf

    port 26379
    bind 0.0.0.0
    daemonize yes
    pidfile "/var/run/redis-sentinel-1.pid"
    logfile "/var/log/redis/sentinel_26379.log"
    dir "/tmp"
    sentinel monitor mymaster 8.129.113.233 6379 2
    sentinel down-after-milliseconds mymaster 5000
    sentinel auth-pass mymaster 123456
    sentinel failover-timeout mymaster 30000

port 26380
bind 0.0.0.0
daemonize yes
pidfile "/var/run/redis-sentinel-2.pid"
logfile "/var/log/redis/sentinel_26380.log"
dir "/tmp"
sentinel monitor mymaster 8.129.113.233 6379 2
sentinel down-after-milliseconds mymaster 5000
sentinel auth-pass mymaster 123456
sentinel failover-timeout mymaster 30000

port 26381
bind 0.0.0.0
daemonize yes
pidfile "/var/run/redis-sentinel-3.pid"
logfile "/var/log/redis/sentinel_26381.log"
dir "/tmp"
sentinel monitor mymaster 8.129.113.233 6379 2
sentinel down-after-milliseconds mymaster 5000
sentinel auth-pass mymaster 123456
sentinel failover-timeout mymaster 30000

• Remember to create the / var/log/redis folder
• Start sentinel cluster

  ./redis-server /usr/local/redis/conf/sentinel-1.conf --sentinel
  ​
  ./redis-server /usr/local/redis/conf/sentinel-2.conf --sentinel
  ​
  ./redis-server /usr/local/redis/conf/sentinel-3.conf --sentinel

• The network security group requires open ports

• advantage

  • The master and slave can switch automatically, with higher availability

• shortcoming

  • The master-slave switch will lose transient data
  • The write capacity and storage capacity of the primary node are limited

III. SpringBoot / microservice cloud integration Redis master-slave + sentinel Sentinel

• Comment out host and port
• New configuration

  sentinel:
    master: mymaster
    nodes: 8.129.113.233:26379,8.129.113.233:26380,8.129.113.233:26381