concept
Master slave replication refers to copying data from one Redis server to other Redis servers.
The former is called master / leader and the latter is called slave / follower. Data replication is unidirectional and can only be from master node to slave node.
Master mainly writes, Slave mainly reads.
By default, each Redis server is a master node, and a master node can have zero or more slave nodes, but a slave node can only have one master node.
Master - slave copy, read - write separation
effect
- Data redundancy: master-slave replication realizes the hot backup of data, which is a data redundancy method other than persistence;
- Fault recovery: when the master node has a problem, the slave node can provide services to achieve fast fault recovery; In fact, it is a kind of service redundancy;
- Load balancing: on the basis of master-slave replication, combined with read-write separation, the master node can provide write services, The slave node provides the read service (that is, the application connects the master node when writing Redis data, and the application connects the slave node when reading Redis data) to share the server load; especially in the scenario of less writing and more reading, sharing the read load through multiple nodes can greatly improve the parallel traffic of Redis server;
- Cornerstone of high availability (cluster): in addition to the above functions, master-slave replication is also the basis for sentinel and cluster implementation. Therefore, master-slave replication is the basis for Redis high availability.
Generally speaking, to apply Redis to engineering projects, only one Redis cannot be used (downtime). The reasons are as follows:
-
Structurally, a single Redis server will have a single point of failure, and one server needs to handle all request loads, resulting in great pressure;
-
In terms of capacity, the memory capacity of a single Redis server is limited. Even if the memory capacity of a Redis server is 256GB, all memory can not be used as Redis storage memory,
Generally speaking, the maximum memory used by a single Redis should not exceed 20GB.
Environment configuration
Configure only slave libraries, not master libraries
127.0.0.1:6379> info replication # Replication role:master # Role: master connected_slaves:0 #No slave master_replid:0d064ffd4a54a85fdd85637328e88d4abd05d8ac master_replid2:0000000000000000000000000000000000000000 master_repl_offset:0 second_repl_offset:-1 repl_backlog_active:0 repl_backlog_size:1048576 repl_backlog_first_byte_offset:0 repl_backlog_histlen:0 127.0.0.1:6379>
Because I only use one server here, we configure a pseudo cluster environment and modify the corresponding configuration information
- Port: the port number occupied by the process;
- pidfile: record the ID of the process. The file has a lock, which can prevent multiple starts of the program;
- logfile: specify the location of the log file;
- dbfilename: persistent file location
After modifying the configuration information, start three Redis services respectively according to the configuration file
One master and two slaves
By default (before configuration), each Redis server is the master node;
Configure only slave libraries, not master libraries
Master recognition: one master (6379) and two slaves (6380, 6381)
The host can read and write, and the slave can only read and cannot write
Important!!!
If the master node server has a password, as the slave node server, it should be added in the configuration file
masterauth　xxx(Master node connection password)
For example:
6380 port server, configuration file redis80 Conf, search the configuration parameter masterauth, and modify it to
masterauth 6379
Before configuration:
127.0.0.1:6380> info replication # Replication role:master connected_slaves:0 master_replid:f89b7907749c026069c80059a9d2cd41fae23e39 master_replid2:0000000000000000000000000000000000000000 master_repl_offset:0 second_repl_offset:-1 repl_backlog_active:0 repl_backlog_size:1048576 repl_backlog_first_byte_offset:0 repl_backlog_histlen:0
Configure the slave in command mode
# Command host address host port 127.0.0.1:6380> SLAVEOF 127.0.0.1 6379 OK
Information after slave configuration:
127.0.0.1:6380> info replication # Replication role:slave master_host:127.0.0.1 master_port:6379 master_link_status:up master_last_io_seconds_ago:10 master_sync_in_progress:0 slave_repl_offset:14 slave_priority:100 slave_read_only:1 connected_slaves:0 master_replid:2382336ee0bd6abe27c267b35c827bcecbb3fa91 master_replid2:0000000000000000000000000000000000000000 master_repl_offset:14 second_repl_offset:-1 repl_backlog_active:1 repl_backlog_size:1048576 repl_backlog_first_byte_offset:1 repl_backlog_histlen:14
Host information:
127.0.0.1:6379> info replication # Replication role:master connected_slaves:1 # More slave information slave0:ip=127.0.0.1,port=6380,state=online,offset=798,lag=1 master_replid:2382336ee0bd6abe27c267b35c827bcecbb3fa91 master_replid2:0000000000000000000000000000000000000000 master_repl_offset:798 second_repl_offset:-1 repl_backlog_active:1 repl_backlog_size:1048576 repl_backlog_first_byte_offset:1 repl_backlog_histlen:798
Configuration file configuration slave
The real slave master configuration should be configured in the configuration file so that it is permanent. We use the command here to configure, which is temporary.
To view a profile as a slave node:
- Configure the ip address and port number of the host. At this time, starting the server is the existence of the slave node;
- Host password
Host disconnected
The master server is turned off and the slave still exists, but the connection status is master_link_status changes to down.
When the host server restarts, the master-slave relationship still exists.
Slave disconnect
After No. 1 slave is disconnected and restarted, No. 1 slave becomes a new host (the host information is not configured in the configuration file, but the master-slave relationship is established by command);
After slave 1 establishes the master-slave relationship with the host again, slave 1 can still query the data newly written by the host during the disconnection of slave 1.
Replication principle
After Slave is successfully started and connected to the master, it will send a sync synchronization command
After receiving the command, the master starts the background save process and collects all the received commands for modifying the dataset. After the background process is executed, the master will transfer the whole data file to the slave and complete a complete synchronization;
Full copy: after receiving the database file data, the Slave service saves it and loads it into memory;
Incremental copy: the Master continues to transmit the modification commands of all new mobile phones to Slave at one time to complete synchronization;
However, as long as the Master is reconnected, a full synchronization (full replication) will be performed automatically!
Sugar gourd model
A: Host;
B: Slave;
C: Slave;
A is the host of B, and B is the host of C. In a sense, B is both a slave and a host. However, using the command info replication, you can see that B is still a slave. It can only read but not write.
There has always been a problem with master-slave replication, that is, when the host goes down, all its slaves have little meaning. They can only read, not write, and cannot store new data;
So can you choose one of its slave machines as the host after the host goes down?
Conspire to usurp the throne
Manual
When the host does not exist, we select a slave and enter the command manually:
SLAVEOF no one
Change the slave into a master, and then manually operate other slaves to connect to the latest master node.
SLAVEOF 127.0.0.1 6388
Sentinel mode
Method of master-slave switching technology: when the master server is down, a slave server needs to be manually switched to the master server, which requires manual intervention, time-consuming and laborious, and the service will not be available for a period of time. This is not a recommended way. More often, we give priority to sentinel mode. Redis has officially provided sentinel architecture since 2.8 to solve this problem.
The automatic version of Mou Chao's usurpation can monitor whether the host fails in the background. If it fails, it will automatically convert from the library to the main library according to the number of votes.
principle
Sentinel mode is a special mode. Firstly, Redis provides sentinel commands. Sentinel is an independent process. As a process, it will run independently.
The principle is that the sentinel sends a command and waits for the response of the Redis server, so as to monitor multiple running Redis instances.
The sentry here has two functions:
- Send commands to let Redis server return to monitor its running status, including master server and slave server;
- When the sentinel detects that the Master is down, it will automatically switch a Slave to the Master, and then notify other Slave servers through publishing and subscription to modify the configuration file and let them switch the host.
Multi sentry mode
If a sentinel process monitors the Redis server, there may be problems. Therefore, we can use multiple sentinels for monitoring. Each sentinel will also be monitored, which forms a multi sentinel mode.
- Assuming that the host is down, sentry 1 detects this result first, and the system will not fail over immediately. Sentry 1 subjectively thinks that the main server is unavailable, which is a subjective offline phenomenon;
- When the following sentinels also detect that the primary server is unavailable and the number reaches a certain value, a vote will be held between sentinels. The vote will be initiated by one sentinel for failover operation;
- After the switch is successful, each sentinel will switch its monitored slave server to the host through the publish and subscribe mode. This process is called objective offline.
to configure
-
Create sentinel Conf file
-
configuration information
# sentinel monitor sentinel name monitors the host IP address and port number setting (the failover operation is carried out only when how many sentinels monitor the subjective offline of the host) sentinel monitor myredis 127.0.0.1 6379 1 # If the host has a login password, you need to configure the following information # Sentinel auth pass sentinel name host password (Note: because sentinel cannot set different passwords for master and slave respectively, the passwords of master and slave should be the same.) sentinel auth-pass myredis 243600
-
Start redis Sentinel
[root@wrkq1njujislh7mx redis]# ./bin/redis-sentinel sentinel.conf
Simulate host downtime
-
Now we turn off the host
127.0.0.1:6379> shutdown not connected> exit [root@wrkq1njujislh7mx redis]#
-
Then observe the status of its two slaves (try to be fast here, otherwise the Sentry will have changed the host)
-
Wait a moment and we can see that the sentry's log file has changed:
-
At this time, the sentry has selected a new host, the Redis server with port number 6380. Let's check the status of 6380 and 6381 servers again:
-
At this time, the new master-slave relationship has been re established. What if the previous host 6379 comes back again?
[root@wrkq1njujislh7mx redis]# ./bin/redis-server redis79.conf [root@wrkq1njujislh7mx redis]# ./bin/redis-cli -p 6379 -a 243600 Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. 127.0.0.1:6379> ping PONG
Check the status of the server (if I input the command slowly, I can see that the server is still the Master at this time. When the sentry reacts, the server becomes the slave)
At this time, the server automatically changes to 6380 slave, sentinel mode rule.
127.0.0.1:6379> info replication # Replication role:slave master_host:127.0.0.1 master_port:6380 master_link_status:down master_last_io_seconds_ago:-1 master_sync_in_progress:0 slave_repl_offset:1 master_link_down_since_seconds:1628239308 slave_priority:100 slave_read_only:1 connected_slaves:0 master_replid:6de2947f7c68080c6bce977a0b4fca378ceb982b master_replid2:0000000000000000000000000000000000000000 master_repl_offset:0 second_repl_offset:-1 repl_backlog_active:0 repl_backlog_size:1048576 repl_backlog_first_byte_offset:0 repl_backlog_histlen:0
View sentry log:
advantage
- Sentinel cluster, based on master-slave replication, includes all the advantages of master-slave configuration;
- The master-slave can be switched and the fault can be transferred, so the system availability will be better;
- Sentinel mode is the upgrade of master-slave replication mode. From manual to automatic, the whole architecture is more robust.
shortcoming
- Redis is not easy to expand online. Once the cluster capacity reaches the upper limit, online expansion will be very troublesome;
- The configuration of sentinel mode is complex and cumbersome (the above configuration information is only a simple configuration for sentinel mode operation)