Catalog
1.1. 1 Common Synchronization Methods
1.1. 2. rsnyc synchronization mode
1.1. 3. rsnyc port and running mode
Introduction to rsnyc
Overview of 1.1
rsync is an open source, fast backup tool that mirrors the entire directory tree between hosts, supports incremental backups, maintains links and privileges, and performs compression before transmission using an optimized synchronization algorithm. Therefore, rsync is ideal for remote backups, mirroring servers, and other applications that are very flexible in achieving file transfer and replication.
In remote synchronization tasks, the client responsible for initiating the rsync synchronization operation is called the initiator, and the server responsible for responding to the rsync synchronization operation from the client is called the synchronization source. During synchronization, the synchronization source is responsible for providing the original location of the file, and the initiator should have read access to that location.
The backup migration feature of the data can be achieved without changing the attribute information of the original data. rsync software is suitable for many operating system platforms such as unix/linux/windows
1.1. 1 Common Synchronization Methods
Full backup: It is inefficient to transfer all the original data and transfer all the original files and new files together in a unified way.
Incremental backup: Incremental replication of different data over the network is efficient by comparing your data with mine through some algorithms before data transfer
1.1. 2. rsnyc synchronization mode
When rsnyc receives data request synchronization, rsnyc divides the data into blocks and compresses them.
1.1. 3. rsnyc port and running mode
rsync Monitor Ruikou: 873
* rsync mode of operation: c / s
Common 1.2 rsync commands
Command format: rsync [options] [original location] target location
Common options: a v z
Command format: rsync-avz username@host address:: shared module name
| -r | Recursive mode, including all files in directories and subdirectories |
| -l | Copy symbolic link files as symbolic link files |
| -v | Show details of the synchronization process |
| -z | Compress when transferring files |
| -a | Archive mode, recursive and preserving object properties |
| -p | Keep permission flags for files |
| -t | Keep time stamps for files |
| -g | Keep group tags for files (superuser only) |
| -o | Keep the owner tag of the file (superuser only) |
| -H | Keep Hard Linked Files |
| -A | Preserve ACL attribute information |
| -D | Keep device files and other special files |
| --delete | Delete files that have destination location but not original location |
| --checksum | Decide whether to skip file 0.2 based on the object's checksum |
2. Set up rsync
2.1 Experimental Environment
rsync server: 192.168. 162.100
Client: 192.168. 162.111
2.2 Experimental Setup
##rsync Server and Client Shutdown Firewall [root@localhost ~]# systemctl stop firewalld.service [root@localhost ~]# setenforce 0
##rsnyc server installation updates it (default installation) [root@lppdff ~]# rpm -q rsync [root@lppdff ~]# yum install -y rsync ##Modify Profile [root@lppdff ~]# vim /etc/rsyncd.conf uid = root gid = root use chroot = yes #Confined in Source Directory address = 192.168.162.111 #Listening Address port 873 #Listening port number tcp/udp 873 log file = /var/log/rsyncd.log #Log Address pid file = /var/run/rsyncd.pid #File location to store process ID hosts allow = 192.168.162.0/24 #Allowed Client Address [ftp] #Shared Module Name path = /var/www/html #Actual path to original directory comment = web service #Remarks read only = yes #readOnly dont comperss = *.gz *.bz2 *.tgz *.zip *.rar *.z #File types that are no longer compressed during synchronization auth users = backuper #Authorized accounts, multiple accounts separated by spaces secrets file = /etc/rsyncd_users.db
#Create data files for backup accounts [root@lppdff ~]# vim /etc/rsyncd_users.db backuper:123123 #Establish a synchronized user password #Permission set to 600 [root@lppdff ~]# chmod 600 /etc/rsyncd_users.db ##Create Actual Directory [root@lppdff ~]# mkdir -p /var/www/html [root@lppdff ~]# chmod +r /var/www/html [root@lppdff ~]# ls -ld /var/www/html ##Start rsync [root@lppdff ~]# rsync --daemon ##Check to see if the service is started [root@lppdff ~]# netstat -anpt |grep rsync ##Shut down services #Kill Process [root@lppdff ~]# kill $(cat /var/run/rsyncd.pid) #Delete Files [root@lppdff ~]#rm -rf /var/run/rsyncd.pid
Validation
##No interaction [root@localhost ~]# echo "123123" > /etc/server.pass [root@localhost ~]# chmod 600 /etc/server.pass
##delete [root@localhost qwe]# rsync -avz --delete --password-file=/etc/server.pass backuper@192.168.162.100::ftp /qwe
Scenario: When a client creates a new file in a synchronized folder, what happens if the client requests synchronization again at this time
[root@localhost ~]# cd /qwe/
[root@localhost qwe]# ls
a.txt
[root@localhost qwe]# touch b.txt
[root@localhost qwe]# ls
a.txt b.txt
[root@localhost qwe]# rsync -avz --delete --password-file=/etc/server.pass backuper@192.168.162.100::ftp /qwe
receiving incremental file list
deleting b.txt
./
sent 60 bytes received 111 bytes 31.09 bytes/sec
total size is 0 speedup is 0.00
[root@localhost qwe]#
If deleted at this time, it will be deleted if it has priority over what the source target has and what the target does not have.
3. inotify auxiliary tools
Introduction to 3.1 Tools
Monitor file system changes and respond with notifications
Adjust inotify kernel parameters (optimization)
Etc/syscti. Conf (Kernel Parameter Profile)
| inotifywait | #For continuous monitoring, real-time output of results |
| inotifywatch | #For short-term monitoring, output results when the task is complete |
| max_queue_events | #Monitor event queue size |
| max_user_instances | #Maximum number of monitoring instances |
| max_user_watches | #Maximum number of monitoring files per instance |
3.2 Intifywait command format
inotifywait [parameter]
| m | Continuous monitoring |
| r | Recursively monitor all child objects |
| q | Simplify output information |
| e | Specify which event types to monitor |
##rsnyc server shutdown read-only [root@lppdff html]# vim /etc/rsyncd.conf read only = no #Modify read-only mode to no ##Close process [root@lppdff html]# kill $(cat /var/run/rsyncd.pid) ##Restart Process [root@lppdff html]# rsync --daemon [root@lppdff html]# netstat -natp |grep rsync
##Client adjusts inotify kernel parameters [root@localhost ~]#cat /proc/sys/fs/inotify/max_queued_events 16384 [root@localhost ~]#cat /proc/sys/fs/inotify/max_user_instances 128 [root@localhost ~]#cat /proc/sys/fs/inotify/max_user_watches 8192 ##Add parameters and refresh [root@localhost qwe]# vi /etc/sysctl.conf fs.inotify.max_queued_events = 32768 fs.inotify.max_user_instances = 1024 fs.inotify.max_user_watches = 1048576 [root@localhost qwe]# sysctl -p [root@localhost qwe]# ##Clients place installation packages in [root@localhost qwe]# cd /opt/ [root@localhost opt]# rz -E rz waiting to receive. ##Install Compilation Environment [root@localhost opt]# yum install -y gcc gcc-c++ ##Unpack Compilation [root@localhost opt]#tar xf inotify-tools-3.14.tar.gz [root@localhost opt]#cd inotify-tools-3.14/ [root@localhost inotify-tools-3.14]#./configure [root@localhost inotify-tools-3.14]#make && make install
Open two clients to perform validation
[root@localhost ~]# inotifywait -mrq -e modify,create,move,delete /qwe ##You can execute the "inotifywait" command first, then open a new terminal to add files to/qwe directory, move files, and track screen output results in the original terminal
##Write trigger synchronization scripts
#!/bin/bash
INOTIFY_CMD="inotifywait -mrq -e create,delete,move,modify,attrib /qwe"
RSYNC_CMD="rsync -azH --delete --password-file=/etc/server.pass /var/www/html backuper@192.168.162.100::ftp"
$INOTIFY_CMD | while read DIRECTORY EVENT FILE
do
if [ $(pgrep rsync | wc -l) -le 0 ] ; then
$RSYNC_CMD
# echo "${FILE} was rsynced" >>/opt/inotify_rsync.log
fi
done
Validation
