What is rsync, its principle and preliminary construction have been introduced earlier. Here I will add the rest, real-time remote synchronization of rsync.
1, What is real-time synchronization
Real time synchronization uses the inotify notification interface to monitor various changes of the file system. The changes of the system include the following: file access, deletion, movement, modification, etc.
Using this mechanism, it is very convenient to realize the alarm and incremental backup of file changes, and respond to directories or files in time.
Combining inotify mechanism with rsync can realize triggered backup, that is, real-time synchronization. That is, as long as the document in the original location changes, the incremental backup operation is started immediately, otherwise it is in a silent waiting state. In this way, the delay and cycle density of backup according to fixed cycle are avoided
2, Construction process
1. Server
-
Update rsync first
yum install rsync -y
-
Enter the configuration file for modification
vim /etc/rsyncd.conf
-
Meaning of each command line
uid = nobody #The default is nobody, which indicates the permissions of the shared directory that can be used and the user identity used gid = nobody #The default is nobody, which shows the permissions of the shared directory that can be used and the user identity used use chroot = yes #The user's permissions are limited according to the environment. The original permissions of the user will be confined in this directory, and the permissions of other directories will be degraded address = 192.168.75.51 #The listening address. The default is to fill in your own local IP address port 873 #The listening port is 873 by default log file = /var/log/rsyncd.log #log file location pid file = /var/run/rsyncd.pid #pid log file location hosts allow = 192.168.75.0/24 #Allowed client addresses [wwwroot] #Called synchronization module path = /var/www/html #The path to the synchronization file comment = Document Root of www.ljm.com #You can modify it or not read only = yes #Whether to read only. The default value is yes. You can directly default it dont compress = *.gz *.tgz *.zip *.z *.rar *.rpm *.deb *.bz2 #A format that no longer requires compression auth users = backuper #Users logged in to this module secrets file = /etc/rsyncd_users.db #Configure the password for the user accessing the module
-
Enter the user password file to configure the password
vim /etc/rsyncd_users.db #Enter the configuration file and insert the following statement backuper:123123 #User name: password
-
Give the user permission to execute the password file
chmod 600 /etc/rsyncd_users.db
-
Create shared directory
mkdir -p /var/www/html #create a file chmod +r /var/www/html/ #Give directory read permission
-
Open service
rsync --daemon #Open service netstat -natp | grep rsync #View port details
-
Temporarily shut down the service first
kill 3062 #Followed by my own process number
Create a file under / var/www/html /
2. Enter client
- Synchronize
rsync -avz backuper@192.168.75.51::wwwroot /lhk
[the external chain image transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-cX5G7b5T-1640832049263)(C:\Users \ Li Hongkun \ appdata \ roaming \ typora user images \ image-20211230084801205. PNG)]
As shown in the figure, the file synchronized to
- Face interaction synchronization mode
echo '123123' > /etc/rsyncd_users.db #Write the password into the file with the same name as the synchronization source chmod 600 /etc/rsyncd_users.db #change permission
- Face interactive synchronization file
rsync -avz --password-file=/etc/rsyncd_users.db backuper@192.168.75.51::wwwroot /lhk
3. Real time synchronization construction
-
Enter the synchronization source to configure parameters
vim /etc/rsyncd.conf #Enter the configuration file and close it read-only read only = no #After closing, save and exit kill `cat /var/run/rsyncd.pid` #Kill the process first Restart again rsync --daemon #start-up
-
Go to the initiator and set the configuration parameters
cat /proc/sys/fs/inotify/max_user_instances 128 #The default value is 128 cat /proc/sys/fs/inotify/max_user_watches 8192 #The default value is 8192 cat /proc/sys/fs/inotify/max_queued_events 16384 #The default value is 16384 #Enter the above three commands to view the inotify kernel parameters #Write kernel parameters to a file vim /etc/sysctl.conf fs.inotify.max_queued_events = 16384 #Monitors the size of the transaction queue fs.inotify.max_user_instances = 128 #Maximum number of monitored instances fs.inotify.max_user_watches = 8192 #How many files can each instance monitor at most #After insertion, save and exit to view the addition sysctl -p #View added information fs.inotify.max_queued_events = 16384 fs.inotify.max_user_instances = 128 fs.inotify.max_user_watches = 8192 The above is the output result. Compare it with the value you entered. Is there any error
-
Initiator installation inotify
yum install -y gcc gcc-c++ #Install dependent environment first tar zxvf inotify-tools-3.14.tar.gz #Unzip the installation package cd inotify-tools-3.14/ #Enter file ./configure #Compile first make && make install #make installation file
-
Open a new initiator (this one is OK, just open a new connection)
inotifywait -mrq -e modify,create,move,delete /lhk #Initiate monitoring -e, real-time monitoring, followed by the type of monitoring. Listening directory
-
Test
touch 2.2 #To create a file [root@server ~]# inotifywait -mrq -e modify,create,move,delete /lhk /lhk/ CREATE 2.2 #After you go to the listening machine, you will find that the operation steps of creating a file have appeared /lhk/ CREATE .2.2.swp /lhk/ CREATE .2.2.swx /lhk/ DELETE .2.2.swx /lhk/ DELETE .2.2.swp /lhk/ CREATE .2.2.swp /lhk/ MODIFY .2.2.swp /lhk/ MODIFY .2.2.swp #These log information will be generated when accessing the file /lhk/ CREATE 4913 /lhk/ DELETE 4913 /lhk/ MOVED_FROM 2.2 /lhk/ MOVED_TO 2.2~ /lhk/ CREATE 2.2 /lhk/ MODIFY 2.2 /lhk/ MODIFY .2.2.swp /lhk/ DELETE 2.2~ /lhk/ DELETE .2.2.swp These are the log information generated after modifying the file
-
Create synchronization script
vim /opt/inotify.sh
#!/bin/bash
INOTIFY_CMD="inotifywait -mrq -e create,delete,move,modify,attrib /lhk/"
#Functions for listening
RSYNC_CMD="rsync -apzH --delete --password-file=/etc/rsyncd_users.db /lhk/ backuper@192.168.75.51::wwwroot/"
#Functions for synchronization
$INOTIFY_CMD | while read DIRECTORY EVENT FILE
#Transfer the value monitored by the parameter to the later, and traverse the following types of logs
do
if [ $(pgrep rsync | wc -l) -le 0 ] ; then
#After the change, it will detect whether synchronization is in progress. If synchronization is in progress, no operation will be performed. If no operation is performed, the following parameters will be executed.
$RSYNC_CMD
# echo "${FILE} was rsynced" >>/opt/inotify_rsync.log
fi
done
-
Modify file permissions
chmod +x /opt/inotify.sh #Add execution permissions to script files chmod +x /etc/rc.d/rc.local #Boot file, add execution permission echo '/opt/inotify.sh' >> /etc/rc.d/rc.local #Add to boot file sh -x /opt/inotify.sh #Execute script
-
Test after opening
As above, open another connection to the monitoring directory to create a new file touch 1.1 After the file is created, the synchronization operation is displayed. + rsync -apzH --delete --password-file=/etc/rsyncd_users.db /lhk/ backuper@192.168.75.51::wwwroot/ + read DIRECTORY EVENT FILE ++ wc -l ++ pgrep rsync + '[' 0 -le 0 ']' + rsync -apzH --delete --password-file=/etc/rsyncd_users.db /lhk/ backuper@192.168.75.51::wwwroot/ To the script, fill in the automatic synchronization server to view [root@rsync html]# ls [root@rsync html]# #As you can see, it is now an empty directory Then go back to the server that started the script to create the service [root@server lhk]# touch 1.1 [root@server lhk]# ls 1.1 #After the creation is completed, wait a moment and return to the server specified in the script [root@rsync html]# ls 1.1 #This server has not performed any artificial creation operation, which means that the synchronization has been successfully performed
Summary
rsync is a very practical module with a wide range of application scenarios. It can be used to remotely synchronize files, delete their own or target files based on synchronization, so as to achieve fast and safe deletion, as well as data migration.