Redis Learning Cluster
Preface
In the first place, we learned the basic operation of Redis, the use of Jedis and the persistence scheme of Redis. Next, we learned the cluster management of Redis.
copy
When data is stored on a server, if the server hangs up, the data will be lost, so Redis provides the function of replication to automatically synchronize data on one server to other servers.
In replication operations, databases can be divided into two categories: master and slave. The master database is used for read and write operations, while the slave database is generally used for read operations only and receives synchronized data from the master database.
In Redis, it is very simple to enable replication by configuring slave of MASTER_HOST MASTER_PORT from the database, and without any other configuration, Redis automatically synchronizes changes from the database to the main database.
Of course, if the master library configures the password, you need to configure the master auth PASSWORD from the slave library to configure the password. Otherwise, the slave library cannot synchronize data from the master library.
Once the configuration is complete, you can log in to the slave library and view the replicated information through the command: info replication, as follows
# Replication role:slave master_host:127.0.0.1 master_port:8080 master_link_status:up # Other information slave_read_only:1 master_replid:c1fafc68662e4b495e7a8619a1408ded499dba4e master_replid2:0000000000000000000000000000000000000000 master_repl_offset:14 # Other information
View the corresponding information in the main library
# Replication role:master connected_slaves:1 slave0:ip=127.0.0.1,port=6379,state=online,offset=210,lag=1 master_replid:c1fafc68662e4b495e7a8619a1408ded499dba4e master_replid2:0000000000000000000000000000000000000000 master_repl_offset:210 second_repl_offset:-1
From the above information, we can see that the EDIS of port 6379 is the slave server of port 8080. At this time, login to the 6379 server, you can get the data of 8080, and the data changes of 8080 will automatically synchronize to 6379.
By default, Redis slave libraries only support read-only operations. If you want to write directly to slave libraries, you can open slave-read-only yes/no from slave libraries. However, it should be noted that changes from slave libraries will not be synchronized to master libraries, so in general, we do not configure slave-read-only yes/no from slave libraries.
If you want to disconnect the master-slave relationship, you can use the slave of no one command to achieve the purpose at run time, or slave of HOST PORT to disconnect the current master library and connect to other master libraries.
principle
When a slave library starts up, it will send SYNC commands to the master library. When the master library receives the commands, it will save the snapshot in the background and cache the commands during the snapshot. After the snapshot is completed, the snapshot and the caching commands during the snapshot are sent to the slave library, and the data is loaded into memory from the file by the slave library.
After Redis 2.8, supporting incremental data transmission can greatly improve the transmission performance.
Graph structure
Slave database can not only receive synchronous data from master database, but also exist as master database to form similar graph structure. At the same time, if write mode is opened from database, the data written from database will be synchronized to slave database from database, but will not be written to its peer or parent node.
Persistence
The performance of the main database can be maximized by setting up that the main database does not enable persistence and the persistence function is enabled from the database.
When the primary database crashes, the following steps are needed to recover data from the database
- Using slave of one in slave database will upgrade from database to main database
- The primary database that crashed before startup is then set to the slave database of the new primary database using slave of HOST PORT
It is important to avoid restarting the master database directly. Otherwise, because the master database is not persistent and the data is empty, the state will be passed directly to the slave database, resulting in all data lost.
No Hard Disk Replication
After Redis 2.8, there is no hard disk replication option. When data is synchronized between master and slave, it is transmitted directly through the network instead of being written to the hard disk, thus avoiding additional overhead due to the performance problems of the hard disk. repl-disless-sync yes/no
Sentry
Redis implements automated system monitoring and fault recovery by introducing sentry tools.
Sentinel is an independent process. In a master-slave system, multiple sentries can be used to monitor tasks to ensure that the system is robust enough.
Enabling mode
Write a configuration file named sentinel.conf with a random name
Content is
sentinel monitor mymaster HOST PORT NUM
Among them, mymaster can be arbitrary, according to naming specifications, HOST is the address of the main library, PORT is the port of the main library, and the following NUM is when the main library hangs, how many slave libraries need to vote to become the number of main libraries.
Then start it with redis-sentinel/PATH/TO/SENTINEL_CONFIG tool
Note that if the master library has a configuration password, you need to add the following
sentinel auth-pass mymaster PASSWORD
The output after startup is as follows
19825:X 26 Sep 15:13:33.288 # Sentinel ID is 75c6f08db6fa065e59f161dde3a4be78fb886dfa 19825:X 26 Sep 15:13:33.288 # +monitor master mymaster 127.0.0.1 8080 quorum 1 19825:X 26 Sep 15:13:33.288 * +slave slave 127.0.0.1:8089 127.0.0.1 8089 @ mymaster 127.0.0.1 8080 19825:X 26 Sep 15:13:33.294 * +slave slave 127.0.0.1:6379 127.0.0.1 6379 @ mymaster 127.0.0.1 8080
As you can see, when the Sentinel is activated, the Sentinel will automatically monitor the master database and discover the slave database.
At this point, we simulate the situation that the following main library hangs, and send shutdown command to the main library through cli. After a moment, we can see the following output content.
19825:X 26 Sep 15:14:38.007 # +sdown master mymaster 127.0.0.1 8080 19825:X 26 Sep 15:14:38.007 # +odown master mymaster 127.0.0.1 8080 #quorum 1/1 19825:X 26 Sep 15:14:38.007 # +new-epoch 1 19825:X 26 Sep 15:14:38.007 # +try-failover master mymaster 127.0.0.1 8080 19825:X 26 Sep 15:14:38.011 # +vote-for-leader 75c6f08db6fa065e59f161dde3a4be78fb886dfa 1 19825:X 26 Sep 15:14:38.011 # +elected-leader master mymaster 127.0.0.1 8080 19825:X 26 Sep 15:14:38.011 # +failover-state-select-slave master mymaster 127.0.0.1 8080 19825:X 26 Sep 15:14:38.111 # +selected-slave slave 127.0.0.1:6379 127.0.0.1 6379 @ mymaster 127.0.0.1 8080 19825:X 26 Sep 15:14:38.111 * +failover-state-send-slaveof-noone slave 127.0.0.1:6379 127.0.0.1 6379 @ mymaster 127.0.0.1 8080 19825:X 26 Sep 15:14:38.166 * +failover-state-wait-promotion slave 127.0.0.1:6379 127.0.0.1 6379 @ mymaster 127.0.0.1 8080 19825:X 26 Sep 15:14:38.270 # +promoted-slave slave 127.0.0.1:6379 127.0.0.1 6379 @ mymaster 127.0.0.1 8080 19825:X 26 Sep 15:14:38.270 # +failover-state-reconf-slaves master mymaster 127.0.0.1 8080 19825:X 26 Sep 15:14:38.321 * +slave-reconf-sent slave 127.0.0.1:8089 127.0.0.1 8089 @ mymaster 127.0.0.1 8080 19825:X 26 Sep 15:14:39.276 * +slave-reconf-inprog slave 127.0.0.1:8089 127.0.0.1 8089 @ mymaster 127.0.0.1 8080 19825:X 26 Sep 15:14:39.276 * +slave-reconf-done slave 127.0.0.1:8089 127.0.0.1 8089 @ mymaster 127.0.0.1 8080 19825:X 26 Sep 15:14:39.337 # +failover-end master mymaster 127.0.0.1 8080 19825:X 26 Sep 15:14:39.337 # +switch-master mymaster 127.0.0.1 8080 127.0.0.1 6379 19825:X 26 Sep 15:14:39.337 * +slave slave 127.0.0.1:8089 127.0.0.1 8089 @ mymaster 127.0.0.1 6379 19825:X 26 Sep 15:14:39.337 * +slave slave 127.0.0.1:8080 127.0.0.1 8080 @ mymaster 127.0.0.1 6379 19825:X 26 Sep 15:15:09.367 # +sdown slave 127.0.0.1:8080 127.0.0.1 8080 @ mymaster 127.0.0.1 6379
As you can see, the sentry finds that the main library is hanging, so the sentry requests to check whether the main library is hanging. After getting the confirmation from other nodes, the sentry begins to elect a new main library (6379), and sets the original main library as the slave Library of the new main library. The specific process can refer to the log above.
When I was working on the above case, I had a strange problem.
Since at first only the master library has passwords and the slave library has no passwords, when the master library hangs up, the slave library will use auth command to connect to the new master library. At this time, because the new master library does not set passwords, there will be errors, resulting in the slave library can not connect to the new slave library.
The solution is to either set the password for each slave library, of course, the password is the same, or do not set the password. I use the first method here.
summary
This section mainly studies Redis's replication and sentry management. The replication function of Redis can make multiple replications and separate reading and writing, which greatly improves the overall performance. Sentinels can effectively monitor and manage nodes. Sentinels will conduct a series of elections and other operations to ensure the overall availability when nodes hang up.