Single point disadvantages and Solutions
redis can be a single machine, a single process, or a cache database. The persistence method can be through rdb and aof. Single machine deployment has three disadvantages: single point of failure; Limited capacity; Too many visits can be stressful.
In order to solve the disadvantages of single machine deployment, the mode of master-slave and master-slave can be adopted.
Active and standby: the host is available, and the standby is not in normal state. When the host fails, it is on the standby set-top;
Master Slave: the slave copies the host data. The host can read and write. The slave is only responsible for reading and sharing the reading pressure;
It can also be split according to business modules. The data of different business modules are placed on different machine nodes. Finally, it is necessary to make high availability and deploy multiple hosts.
The data consistency of multiple machines should be considered for the master-slave or master-slave. The synchronous blocking mode will destroy the availability, because the slave is blocked when synchronizing the host data, and can continue to respond only after the synchronization is successful. If a machine fails to synchronize successfully, it will always block the deadlock and destroy the availability, which is strong consistency;
The synchronous data of the host can be asynchronous, but the data will be lost, which is inconvenient;
The redis host is synchronously deployed to the message queue and then returned. The message queue connects to the slave to synchronize data. This is the final consistency and will improve availability.
During master-slave replication, if the host is down, the system will not be available. A monitoring mechanism can be adopted. Multiple monitors monitor a master node. More than half of the monitors believe that the host is down, so they elect a slave node to continue to serve as the master node. When the client accesses the partition data, it may get data with inconsistent versions, resulting in network partition. What does network partition mean? Network partition refers to these services, which are three machines. When I visit them through the network, different clients get different results. One is led astray by him and the other is led astray by him. Then there are two answers to the whole service. When I visit them through the network, network partition appears. It should be tolerant. Can you tolerate it? In other words, it is not necessarily a bad thing, because it depends on your data. If I have a lot of data, a total of 10 machines need to be registered in the service, and then hundreds of other people want to know which machines can be accessed behind you. In fact, each machine only needs to access one of them.
Then, if there is a partition, there is a communication problem between them. Then, two of the 10 have been registered here, eight have been registered there, or two of the eight have died. In any case, in fact, the data in these 10 sets will be scattered here. Even if their data are inconsistent, they can get different results from the outside world, but they will find one of the results they can use, because they only need one of them to be available. So at this time, in fact, you don't need to do this forced consistency and final consistency for it. There may be brain fissure zoning. You can tolerate it because you don't require the consistency of all his data.
Multiple monitoring, monitoring a host, according to the principle of more than half, the minority obeys the majority, the majority thinks it can, and the majority thinks there is a problem. Odd sets are more suitable and cost-effective.
Master slave replication
1. daemonize introduction
A,redis. The daemon thread in the conf configuration file is NO by default.
B. Daemon is used to specify whether redis should be started as a daemon thread.
2. Daemon set yes or no difference
Daemon: Yes: redis adopts the mode of single process and multithreading. When redis When the option Daemon in conf is set to yes, it means that the daemon mode is turned on. In this mode, redis will run in the background and write the process pid number to redis The conf option is in the file set by pidfile. At this time, redis will always run unless you manually kill the process.
Daemon: No: when the daemon option is set to no, the current interface will enter the redis command line interface. Forced exit or closing the connection tools (putty,xshell, etc.) will cause the redis process to exit
daemonize no logfile # Note out the log and print it to the foreground instead of in the log file appendonly no close Aof backups
redis-server /root/test/6379.conf Start three redis example 5.0 Before version 127.0.0.1:6379> help SLAVEOF SLAVEOF host port summary: Make the server a replica of another instance, or promote it as master. Deprecated starting with Redis 5. Use REPLICAOF instead. since: 1.0.0 group: server 5.0 After version 127.0.0.1:6379> help REPLICAOF REPLICAOF host port summary: Make the server a replica of another instance, or promote it as master. since: 5.0.0 group: server
6379 as the master and 6380 / 6381 as the slave
REPLICAOF 127.0.0.1 6379 80.81 Connect to the host by command 6379 Interface display 26183:M 09 Jan 2022 20:52:23.950 * Ready to accept connections 6380 Request connection follow 26183:M 09 Jan 2022 20:58:36.233 * Replica 127.0.0.1:6380 asks for synchronization 26183:M 09 Jan 2022 20:58:36.233 * Partial resynchronization not accepted: Replication ID mismatch (Replica asked for 'ca35e595a688e8bc2f89cd2878b23fb361e70f30', my replication IDs are 'fd49f94377778985974857e8a4d9a53bfeca37d6' and '0000000000000000000000000000000000000000') 26183:M 09 Jan 2022 20:58:36.233 * Starting BGSAVE for SYNC with target: disk 26183:M 09 Jan 2022 20:58:36.233 * Background saving started by pid 26220 26220:C 09 Jan 2022 20:58:36.235 * DB saved on disk 26220:C 09 Jan 2022 20:58:36.235 * RDB: 0 MB of memory used by copy-on-write 26183:M 09 Jan 2022 20:58:36.315 * Background saving terminated with success 26183:M 09 Jan 2022 20:58:36.315 * Synchronization with replica 127.0.0.1:6380 succeeded 6381 Request connection follow 26183:M 09 Jan 2022 20:59:34.281 * Replica 127.0.0.1:6381 asks for synchronization 26183:M 09 Jan 2022 20:59:34.281 * Partial resynchronization not accepted: Replication ID mismatch (Replica asked for 'e8276fae7804af97815fd65282e3c0dcde853271', my replication IDs are 'b9b7f869e63e7dc110211bbb8079f31aa634e0d0' and '0000000000000000000000000000000000000000') 26183:M 09 Jan 2022 20:59:34.281 * Starting BGSAVE for SYNC with target: disk 26183:M 09 Jan 2022 20:59:34.281 * Background saving started by pid 26224 26224:C 09 Jan 2022 20:59:34.284 * DB saved on disk 26224:C 09 Jan 2022 20:59:34.284 * RDB: 0 MB of memory used by copy-on-write 26183:M 09 Jan 2022 20:59:34.351 * Background saving terminated with success 26183:M 09 Jan 2022 20:59:34.351 * Synchronization with replica 127.0.0.1:6381 succeeded
6380 interface display
[root@iZuf6ir4og87e245nh5ltuZ ~]# redis-server /root/test/6380.conf
26187:C 09 Jan 2022 20:52:29.283 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
26187:C 09 Jan 2022 20:52:29.283 # Redis version=5.0.5, bits=64, commit=00000000, modified=0, pid=26187, just started
26187:C 09 Jan 2022 20:52:29.283 # Configuration loaded
_._
_.-``__ ''-._
_.-`` `. `_. ''-._ Redis 5.0.5 (00000000/0) 64 bit
.-`` .-```. ```\/ _.,_ ''-._
( ' , .-` | `, ) Running in standalone mode
|`-._`-...-` __...-.``-._|'` _.-'| Port: 6380
| `-._ `._ / _.-' | PID: 26187
`-._ `-._ `-./ _.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' | http://redis.io
`-._ `-._`-.__.-'_.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' |
`-._ `-._`-.__.-'_.-' _.-'
`-._ `-.__.-' _.-'
`-._ _.-'
`-.__.-'
26187:M 09 Jan 2022 20:52:29.284 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
26187:M 09 Jan 2022 20:52:29.284 # Server initialized
26187:M 09 Jan 2022 20:52:29.284 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
26187:M 09 Jan 2022 20:52:29.284 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
26187:M 09 Jan 2022 20:52:29.284 * Ready to accept connections
26187:S 09 Jan 2022 20:58:35.632 * Before turning into a replica, using my master parameters to synthesize a cached master: I may be able to synchronize with the new master with just a partial transfer.
26187:S 09 Jan 2022 20:58:35.632 * REPLICAOF 127.0.0.1:6379 enabled (user request from 'id=3 addr=127.0.0.1:38190 fd=7 name= age=323 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=44 qbuf-free=32724 obl=0 oll=0 omem=0 events=r cmd=replicaof')
26187:S 09 Jan 2022 20:58:36.232 * Connecting to MASTER 127.0.0.1:6379
26187:S 09 Jan 2022 20:58:36.232 * MASTER <-> REPLICA sync started
26187:S 09 Jan 2022 20:58:36.232 * Non blocking connect for SYNC fired the event.
26187:S 09 Jan 2022 20:58:36.232 * Master replied to PING, replication can continue...
26187:S 09 Jan 2022 20:58:36.233 * Trying a partial resynchronization (request ca35e595a688e8bc2f89cd2878b23fb361e70f30:1).
26187:S 09 Jan 2022 20:58:36.233 * Full resync from master: b9b7f869e63e7dc110211bbb8079f31aa634e0d0:0
26187:S 09 Jan 2022 20:58:36.233 * Discarding previously cached master state.
26187:S 09 Jan 2022 20:58:36.315 * MASTER <-> REPLICA sync: receiving 175 bytes from master
26187:S 09 Jan 2022 20:58:36.315 * MASTER <-> REPLICA sync: Flushing old data
26187:S 09 Jan 2022 20:58:36.315 * MASTER <-> REPLICA sync: Loading DB in memory
26187:S 09 Jan 2022 20:58:36.316 * MASTER <-> REPLICA sync: Finished with success
6381 interface display
26191:C 09 Jan 2022 20:52:33.169 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
26191:C 09 Jan 2022 20:52:33.169 # Redis version=5.0.5, bits=64, commit=00000000, modified=0, pid=26191, just started
26191:C 09 Jan 2022 20:52:33.169 # Configuration loaded
_._
_.-``__ ''-._
_.-`` `. `_. ''-._ Redis 5.0.5 (00000000/0) 64 bit
.-`` .-```. ```\/ _.,_ ''-._
( ' , .-` | `, ) Running in standalone mode
|`-._`-...-` __...-.``-._|'` _.-'| Port: 6381
| `-._ `._ / _.-' | PID: 26191
`-._ `-._ `-./ _.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' | http://redis.io
`-._ `-._`-.__.-'_.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' |
`-._ `-._`-.__.-'_.-' _.-'
`-._ `-.__.-' _.-'
`-._ _.-'
`-.__.-'
26191:M 09 Jan 2022 20:52:33.170 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
26191:M 09 Jan 2022 20:52:33.170 # Server initialized
26191:M 09 Jan 2022 20:52:33.170 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
26191:M 09 Jan 2022 20:52:33.170 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
26191:M 09 Jan 2022 20:52:33.170 * Ready to accept connections
26191:S 09 Jan 2022 20:59:33.382 * Before turning into a replica, using my master parameters to synthesize a cached master: I may be able to synchronize with the new master with just a partial transfer.
26191:S 09 Jan 2022 20:59:33.382 * REPLICAOF 127.0.0.1:6379 enabled (user request from 'id=3 addr=127.0.0.1:52832 fd=7 name= age=372 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=44 qbuf-free=32724 obl=0 oll=0 omem=0 events=r cmd=replicaof')
26191:S 09 Jan 2022 20:59:34.281 * Connecting to MASTER 127.0.0.1:6379
26191:S 09 Jan 2022 20:59:34.281 * MASTER <-> REPLICA sync started
26191:S 09 Jan 2022 20:59:34.281 * Non blocking connect for SYNC fired the event.
26191:S 09 Jan 2022 20:59:34.281 * Master replied to PING, replication can continue...
26191:S 09 Jan 2022 20:59:34.281 * Trying a partial resynchronization (request e8276fae7804af97815fd65282e3c0dcde853271:1).
26191:S 09 Jan 2022 20:59:34.282 * Full resync from master: b9b7f869e63e7dc110211bbb8079f31aa634e0d0:70
26191:S 09 Jan 2022 20:59:34.282 * Discarding previously cached master state.
26191:S 09 Jan 2022 20:59:34.351 * MASTER <-> REPLICA sync: receiving 175 bytes from master
26191:S 09 Jan 2022 20:59:34.352 * MASTER <-> REPLICA sync: Flushing old data
26191:S 09 Jan 2022 20:59:34.352 * MASTER <-> REPLICA sync: Loading DB in memory
26191:S 09 Jan 2022 20:59:34.352 * MASTER <-> REPLICA sync: Finished with success
The slave is not writable but only readable
127.0.0.1:6381> set k2 3 (error) READONLY You can't write against a read only replica. 127.0.0.1:6381>
rdb saved to disk
26191:S 09 Jan 2022 21:07:34.073 * 1 changes in 900 seconds. Saving... 26191:S 09 Jan 2022 21:07:34.074 * Background saving started by pid 26243 26243:C 09 Jan 2022 21:07:34.076 * DB saved on disk 26243:C 09 Jan 2022 21:07:34.077 * RDB: 0 MB of memory used by copy-on-write 26191:S 09 Jan 2022 21:07:34.174 * Background saving terminated with success
81 disconnected host knows
26183:M 09 Jan 2022 21:10:32.252 # Connection with replica 127.0.0.1:6381 lost.
When starting, specify the following host and open AOF backup
redis-server /root/test/6381.conf --replicaof 127.0.0.1 6379 --appendonly yes
When aof and rdb backups are enabled, the aof file will display the rdb in the upper part, and then the aof log after the time point is added. The rdb file will record the pid that has been followed.
The host is down. I know from the chance.
replica-server-stale-data yes/no
When the slave starts. Time of backup and restore of host data, whether existing data supports query, etc