catalogue
4 advantages and disadvantages of AOF
5.1 comparison between RDB and aof
5.2 enable two persistence methods at the same time
1 what is AOP
Record every write operation in the form of log, and record all write instructions executed by redis (read operation is not recorded). Only the file can be added, but the file cannot be overwritten. Redis will read the file and rebuild the data at the beginning of startup. In other words, if redis restarts, it will execute the write instructions from front to back according to the contents of the log file to complete the data recovery
2 appendonly.aof file
############################## APPEND ONLY MODE ############################### # By default Redis asynchronously dumps the dataset on disk. This mode is # good enough in many applications, but an issue with the Redis process or # a power outage may result into a few minutes of writes lost (depending on # the configured save points). # # The Append Only File is an alternative persistence mode that provides # much better durability. For instance using the default data fsync policy # (see later in the config file) Redis can lose just one second of writes in a # dramatic event like a server power outage, or a single write if something # wrong with the Redis process itself happens, but the operating system is # still running correctly. # # AOF and RDB persistence can be enabled at the same time without problems. # If the AOF is enabled on startup Redis will load the AOF, that is the file # with the better durability guarantees. # # Please check https://redis.io/topics/persistence for more information. appendonly no # The name of the append only file (default: "appendonly.aof") appendfilename "appendonly.aof" # The fsync() call tells the Operating System to actually write data on disk # instead of waiting for more data in the output buffer. Some OS will really flush # data on disk, some other OS will just try to do it ASAP. # # Redis supports three different modes: # # no: don't fsync, just let the OS flush the data when it wants. Faster. # always: fsync after every write to the append only log. Slow, Safest. # everysec: fsync only one time every second. Compromise. # # The default is "everysec", as that's usually the right compromise between # speed and data safety. It's up to you to understand if you can relax this to # "no" that will let the operating system flush the output buffer when # it wants, for better performances (but if you can live with the idea of # some data loss consider the default persistence mode that's snapshotting), # or on the contrary, use "always" that's very slow but a bit safer than # everysec. # # More details please check the following article: # http://antirez.com/post/redis-persistence-demystified.html # # If unsure, use "everysec". # appendfsync always appendfsync everysec # appendfsync no # When the AOF fsync policy is set to always or everysec, and a background # saving process (a background save or AOF log background rewriting) is # performing a lot of I/O against the disk, in some Linux configurations # Redis may block too long on the fsync() call. Note that there is no fix for # this currently, as even performing fsync in a different thread will block # our synchronous write(2) call. # # In order to mitigate this problem it's possible to use the following option # that will prevent fsync() from being called in the main process while a # BGSAVE or BGREWRITEAOF is in progress. # # This means that while another child is saving, the durability of Redis is # the same as "appendfsync none". In practical terms, this means that it is # possible to lose up to 30 seconds of log in the worst scenario (with the # default Linux settings). # # If you have latency problems turn this to "yes". Otherwise leave it as # "no" that is the safest pick from the point of view of durability. no-appendfsync-on-rewrite no # Automatic rewrite of the append only file. # Redis is able to automatically rewrite the log file implicitly calling # BGREWRITEAOF when the AOF log size grows by the specified percentage. # # This is how it works: Redis remembers the size of the AOF file after the # latest rewrite (if no rewrite has happened since the restart, the size of # the AOF at startup is used). # # This base size is compared to the current size. If the current size is # bigger than the specified percentage, the rewrite is triggered. Also # you need to specify a minimal size for the AOF file to be rewritten, this # is useful to avoid rewriting the AOF file even if the percentage increase # is reached but it is still pretty small. # # Specify a percentage of zero in order to disable the automatic AOF # rewrite feature. auto-aof-rewrite-percentage 100 auto-aof-rewrite-min-size 64mb
appendonly no
appendonly no means that aof snapshot storage is not enabled, and appendonly yes means that snapshot storage is enabled
# The name of the append only file (default: "appendonly.aof")
appendfilename "appendonly.aof"
The default AOF snapshot name is appendonly aof
# appendfsync always
appendfsync everysec
# appendfsync no
alaways: every time a data change occurs in synchronous persistence, it will be immediately recorded to the disk. The performance is poor, but the data integrity is good
everysec: Factory asynchronous recommendation, asynchronous operation, recording every second. If the machine goes down within one second, there will be data loss
no:
no-appendfsync-on-rewrite no
Whether Appendfsync can be used during rewriting, and the default no can be used to ensure data security
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
Set overridden base value
Note: appendonly Aof and dump RDB can be placed in the same bin directory, and appendonly will be preferred when starting Aof file.
If redis fails to start at this time, it can be appendonly In the AOF file, due to some network delay, packet loss and other reasons, there are some syntax that redis cannot parse. At this time, you can execute redis check AOF -- fix appendix only The AOF command automatically checks for and corrects incorrect syntax.
3 Rewrite
3.1 what is rewrite
Aof adopts the method of file appending, and the file will become larger and larger. In order to avoid this situation, a rewriting mechanism is added. When the file size of AOF exceeds the set threshold, redis will start the content compression of AOF file, and only the minimum instruction set that can recover the data is reserved. bgrewriteaof can be used
3.2 rewriting principle
When the aof file continues to grow too large, it will fork out a new process to rewrite the file (also write the temporary file first and then rename), traverse the data in the memory of the new process, and each record has a set statement. The operation of rewriting aof file does not read the old aof file, but rewrites a new aof file with the contents of the database in the whole memory in the form of command, which is somewhat similar to snapshot
3.3 trigger mechanism
redis will record the size of the aof when it was last rewritten. The default configuration is triggered when the size of the aof file is twice the size after the last rewrite and the file is greater than 64m
4 advantages and disadvantages of AOF
Advantages: you can flexibly configure synchronization rules
Synchronization per second: appendfsync # ways: synchronization persistence. Every time data change occurs, it will be immediately recorded to the disk. The performance is poor, but the data integrity is good
Synchronization of every modification: appendfsync every sec: Factory asynchronous recommendation, asynchronous operation, recording every second. If the machine goes down within one second, there will be data loss
Out of Sync: appendfsync # no:
Disadvantages:
1) For the data of the same dataset, aof files are much larger than rdb files, and the recovery speed is slower than rdb files
2) The efficiency of aof is slower than rdb, the efficiency of synchronization strategy per second is better, and the asynchronous efficiency is the same as rdb
5 Summary
5.1 comparison between RDB and aof
RDB persistence mode can snapshot and store data within a specified time interval
Aof persistence records every write operation to the server. When the server rewrites, it will re execute these commands to recover the original data. The AOF command additionally saves each write operation to the end of the file with redis protocol. Redis can also rewrite the AOF file in the background, so that the volume of the AOF file will not be too large
If you only want the data to exist when the server is running, you can also choose no persistence method
5.2 enable two persistence methods at the same time
In this case, when redis restarts, AOF files will be loaded first to recover the original data. Because in general, the data saved by AOF file is more complete than that saved by RDB file. RDB data is not real-time. When using both, the server will only find AOF files when restarting (it does not mean that we only use AOF), because RDB is more suitable for fast restart of backup database (AOF is changing and hard to backup), and there will be no potential bug s in AOF.