File synchronization tool
- preview contents
- After class summary
preview contents
10.28 introduction to Rsync tool 10.29/10.30 rsync common options 10.31 rsync synchronization via ssh 10.32/10.33 rsync service synchronization 10.34 linux system log 10.35 screen tool extend
- Linux Log File Manager logrotate http://linux.cn/article-4126-1.html
- xargs usage details http://blog.csdn.net/zhangfn2011/article/details/6776925
1. Introduction to Rsync tool
There are many data backup tools under the Linux system. The commonly used one is rsync, which literally means remote sync. rsync can not only remotely synchronize data (similar to scp), but also locally synchronize data (similar to cp). However, unlike cp or scp, it will not overwrite the previous data (if the data already exists), Instead, first judge the difference between the existing data and the new data. Only when the data is different will the different parts be covered.
rsync is a fast and versatile file replication tool. It can be replicated locally, from any remote shell or from another host to the remote rsync daemon. It provides a number of options to control every aspect of its behavior and allows very flexibility in specifying the set of files to copy. It is famous for its incremental conversion algorithm, which can reduce the amount of data sent by sending far only the difference between the source file on the network and the existing file in the target. rsync is widely used for backup and mirroring, as well as daily use as an improved replication command.
rsync is not included in the system minimization installation, so install rsync first:
[root@linux-02 ~]# yum install -y rsync-3.1.2-4.el7.x86_64 Plug in loaded: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.zju.edu.cn * extras: mirrors.163.com * updates: mirrors.163.com base | 3.6 kB 00:00:00 extras | 3.4 kB 00:00:00 updates | 3.4 kB 00:00:00 Resolving dependencies --> Checking transactions ---> software package rsync.x86_64.0.3.1.2-4.el7 Will be installed --> Resolve dependency complete Dependency resolution =========================================================================================================================== Package framework edition source size =========================================================================================================================== Installing: rsync x86_64 3.1.2-4.el7 base 403 k Transaction summary =========================================================================================================================== Install 1 package Total downloads: 403 k Installation size: 815 k Downloading packages: rsync-3.1.2-4.el7.x86_64.rpm | 403 kB 00:00:04 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : rsync-3.1.2-4.el7.x86_64 1/1 Verification in progress : rsync-3.1.2-4.el7.x86_64 1/1 already installed: rsync.x86_64 0:3.1.2-4.el7 complete!
Use rsync command to copy local files to other local directories, - A contains multiple options, and - v option is a visualization process. You can view the size of the sent file, the size of the received file, the speed, the total size of the file, and the accelerated time.
[root@linux-02 ~]# rsync -av /root/a.txt /tmp/1.txt sending incremental file list a.txt sent 372 bytes received 35 bytes 814.00 bytes/sec total size is 282 speedup is 0.69 [root@linux-02 ~]# rsync -av /root/111/ /tmp/111 sending incremental file list ./ sent 43 bytes received 19 bytes 124.00 bytes/sec total size is 0 speedup is 0.00
In addition to copying local files and directories, rsync can also be copied remotely. The target machine must also have rsync installed. The command usage is as follows. Copy the files of this machine to another machine.
[root@linux-02 ~]# rsync -av /root/a.txt root@192.168.141.130:/root/ The authenticity of host '192.168.141.130 (192.168.141.130)' can't be established. RSA key fingerprint is SHA256:QbOsyZRk1yI6nmj7FKmNO8WxmrY9phyOACZtjFylZgA. RSA key fingerprint is MD5:1e:0b:98:01:4c:14:23:25:e9:16:4e:ea:38:30:9b:f4. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '192.168.141.130' (RSA) to the list of known hosts. root@192.168.141.130's password: sending incremental file list a.txt sent 367 bytes received 34 bytes 89.11 bytes/sec total size is 282 speedup is 0.70 [root@linux-02 ~]#
The rsync command can copy local files to the local machine and local files to remote machines. The specific format is as follows:
rsync format SRC source file DEST Target file rsync [OPTION] ... SRC DEST rsync [OPTION] ... SRC [user@]host:DEST rsync [OPTION] ... [user@]host:SRC DEST rsync [OPTION] ... SRC [user@]host::DEST rsync [OPTION] ... [user@]host::SRC DEST
2.rsync common options
option | meaning |
|---|---|
-a | Contains - rtplgoD |
-r | When synchronizing directories, add the - r option similar to that in cp |
-v | Some information is displayed during synchronization to let us know the synchronization process |
-l | Keep soft connection |
-L | When this option is added, the source file will be synchronized when synchronizing the soft link |
-p | Maintain file permission attributes |
-o | Keep file owner |
-g | Keep file group |
-D | Keep device file information |
-t | Keep the time attribute of the file |
–delete | Delete files not in SRC in DEST |
–exclude | Filter the specified files. For example, – exclude "logs" will filter out the files or directories whose file names contain logs and will not be synchronized |
-P | Display the synchronization process, such as rate, which is more detailed than - v |
-u | With this option, if the file in DEST is newer than SRC, it will not be synchronized |
-z | Compression during transmission |
3.rsync synchronizes via ssh
By default, port 22 is used when using remote software to connect to the server. You can use rsync to synchronize files in the following way.
## adopt ssh Take the file from A Stage pusher B machine ##
[root@linux-01 ~]# rsync -avP /root/ceshi/ 192.168.241.89:/tmp/rsync/
The authenticity of host '192.168.241.89 (192.168.241.89)' can't be established.
ECDSA key fingerprint is SHA256:phzPMTk8az1R0dk6tK3c1m00ux7VDz3WGCnSYUbm6Zw.
ECDSA key fingerprint is MD5:e8:bc:cd:c1:ed:6c:68:9e:0e:99:45:a2:1c:05:64:f1.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.241.89' (ECDSA) to the list of known hosts.
root@192.168.241.89's password:
sending incremental file list
created directory /tmp/rsync
./
pass.txt
2,319 100% 0.00kB/s 0:00:00 (xfr#1, to-chk=10/12)
firewalld/
firewalld/firewalld.conf
2,006 100% 979.49kB/s 0:00:00 (xfr#2, to-chk=8/12)
firewalld/lockdown-whitelist.xml
271 100% 132.32kB/s 0:00:00 (xfr#3, to-chk=7/12)
firewalld/helpers/
firewalld/icmptypes/
firewalld/ipsets/
firewalld/services/
firewalld/zones/
firewalld/zones/public.xml
315 100% 153.81kB/s 0:00:00 (xfr#4, to-chk=1/12)
firewalld/zones/public.xml.old
315 100% 153.81kB/s 0:00:00 (xfr#5, to-chk=0/12)
sent 5,815 bytes received 179 bytes 1,332.00 bytes/sec
total size is 5,226 speedup is 0.87
[root@linux-01 ~]#
## adopt ssh Take the file from B The machine is pulled back A machine ##
[root@linux-01 ~]# rsync -avP 192.168.241.89:/tmp/rsync/pass.txt ./
root@192.168.241.89's password:
receiving incremental file list
pass.txt
2,319 100% 2.21MB/s 0:00:00 (xfr#1, to-chk=0/1)
sent 43 bytes received 2,413 bytes 701.71 bytes/sec
total size is 2,319 speedup is 0.94
[root@linux-01 ~]# ls
anaconda-ks.cfg ceshi pass.txt
[root@linux-01 ~]# How to operate if the ssh of the service is not port 22? You need to add an - e option and add the ssh -p port number in double quotation marks, so that rsync can be used to synchronize operations.
[root@linux-01 ~]# rsync -avP -e "ssh -p 22" /root/ceshi/ 192.168.241.89:/tmp/rsync/ root@192.168.241.89's password: sending incremental file list sent 341 bytes received 18 bytes 102.57 bytes/sec total size is 5,226 speedup is 14.56 [root@linux-01 ~]#
In fact, ssh -p 22 192.168.241.89 is a remote login to the server. The test is as follows
## stay linux-01 Remote login using commands on the machine linux-02 machine ## [root@linux-01 ~]# ssh -p 22 192.168.241.89 root@192.168.241.89's password: Last login: Sat Apr 20 10:03:05 2019 from 192.168.241.1 welcome to linux! [root@linux-02 ~]#
4.rsync service synchronization
- Solution ideas
- 1. Modify the configuration file / etc / rsyncd. Of machine A conf
- 2. Start the service rsync --daemon
- 3. Transmission format: rsync -av test1/ 192.168.241.88::module/dir/
- Synchronous error reporting: whether the route is the same, whether the port is the same, and whether it is a firewall problem.
rsyncd. Detailed explanation of conf configuration file Port: specifies which port to start rsyncd service. The default is port 873. Log file: Specifies the log file. pid file: Specifies the pid file, which is used for process management operations such as service startup and stop. address: Specifies the IP to start rsyncd service. If your machine has multiple IPS, you can specify one of them to start rsyncd service. If you do not specify this parameter, it is started on all IPS by default. []: specify the module name and customize the contents. Path: Specifies the path where the data is stored. use chroot true|false: indicates that chroot should first go to the directory specified by the path parameter before transferring files. The reason for this is to achieve additional security protection, but the disadvantage is that you need to use the roots permission, and you can't back up the directory file pointed to by the symbolic connection pointing to the outside. By default, chroot value is true. If there are soft connection files in your data, Amin suggests that you set it to false. – port specifies the port max connections: Specifies the maximum number of connections. The default is 0, that is, there is no limit. Read only true | false: if it is true, it cannot be uploaded to the path specified by the module. List: indicates whether the module is listed when the user queries the available modules on the server. Set to true to list and false to hide. uid/gid: specifies which user / group to transfer files as. auth users: Specifies the user name to use when transferring. secrets file: Specifies the password file. If this parameter is not specified, password authentication will not be used. Note that the permission of the password file must be 600. Format: Username: password hosts allow: indicates the host allowed to connect to the module, which can be IP or network segment. If there are multiple hosts, separate them with a space. After auth users and secrets file are set, the client and the server also need to use the user name and password. If you want to bring the password in the command line, you can set a password file rsync -avL test@192.168.133.130::test/test1/ /tmp/test8/ --password-file=/etc/pass The content of / etc/pass is a password, and the permission should be changed to 600
On the server side, add and modify the configuration file on machine A (ip address 192.168.241.88), start rsync service, and view the listening port of the service
[root@linux-01 ~]# vim /etc/rsyncd.conf # Append the following to the configuration file port=873 //Specify the port number of rsync. If you modify it, you need to restart the service log file=/var/log/rsync.log pid file=/var/run/rsyncd.pid address=192.168.241.88 //Address fill in this machine [rsync] path=/tmp/rsync use chroot=true max connections=4 read only=no list=true uid=root gid=root auth users=test secrets file=/etc/rsyncd.passwd hosts allow=192.168.241.88 //You need to fill in the ip address of the source machine transmitted to this machine file. The following error is reported because it is filled in incorrectly. [root@linux-01 ~]# rsync --daemon [root@linux-01 ~]# ps aux | grep rsync root 8954 0.0 0.0 114744 568 ? Ss 10:57 0:00 rsync --daemon root 8964 0.0 0.0 112724 984 pts/1 S+ 10:58 0:00 grep --color=auto rsync [root@linux-01 ~]# [root@linux-01 ~]# netstat -lntp Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 192.168.241.88:873 0.0.0.0:* LISTEN 8954/rsync tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 1/systemd tcp 0 0 192.168.122.1:53 0.0.0.0:* LISTEN 7232/dnsmasq tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 6844/sshd tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN 6849/cupsd tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 7199/master tcp 0 0 127.0.0.1:6010 0.0.0.0:* LISTEN 7516/sshd: root@pts tcp 0 0 127.0.0.1:6011 0.0.0.0:* LISTEN 8196/sshd: root@pts tcp6 0 0 :::111 :::* LISTEN 1/systemd tcp6 0 0 :::22 :::* LISTEN 6844/sshd tcp6 0 0 ::1:631 :::* LISTEN 6849/cupsd tcp6 0 0 ::1:25 :::* LISTEN 7199/master tcp6 0 0 ::1:6010 :::* LISTEN 7516/sshd: root@pts tcp6 0 0 ::1:6011 :::* LISTEN 8196/sshd: root@pts [root@linux-01 ~]#
Machine B synchronizes files to machine A. if it finds an error during synchronization, it will prompt that there is no route. Check whether the ip address of ping machine a can be accessed; You can view the release rules of the firewall as needed; Let's stop firewalld and the firewall of machine a for the time being. In this way, it is found that there are still errors when running the command. One is that the directory in our configuration file is not created, and the other is that the ip address filled in "hosts allow" in the configuration file is wrong. You can modify it.
[root@linux-02 ~]# rsync -av /root/rsync/ 192.168.241.88::rsync/tb89
rsync: failed to connect to 192.168.241.88 (192.168.241.88): No route to host (113)
rsync error: error in socket IO (code 10) at clientserver.c(125) [sender=3.1.2]
## See if you can ping through ##
[root@linux-02 ~]# ping 192.168.241.88
PING 192.168.241.88 (192.168.241.88) 56(84) bytes of data.
64 bytes from 192.168.241.88: icmp_seq=1 ttl=64 time=0.636 ms
64 bytes from 192.168.241.88: icmp_seq=2 ttl=64 time=0.758 ms
^C
--- 192.168.241.88 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1014ms
rtt min/avg/max/mdev = 0.636/0.697/0.758/0.061 ms
[root@linux-02 ~]#
## View firewall rules ##
[root@linux-02 ~]# iptables -nvL
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 ACCEPT udp -- virbr0 * 0.0.0.0/0 0.0.0.0/0 udp dpt:53
0 0 ACCEPT tcp -- virbr0 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:53
0 0 ACCEPT udp -- virbr0 * 0.0.0.0/0 0.0.0.0/0 udp dpt:67
0 0 ACCEPT tcp -- virbr0 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:67
931 86398 ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
2 146 ACCEPT all -- lo * 0.0.0.0/0 0.0.0.0/0
95 14462 INPUT_direct all -- * * 0.0.0.0/0 0.0.0.0/0
95 14462 INPUT_ZONES_SOURCE all -- * * 0.0.0.0/0 0.0.0.0/0
95 14462 INPUT_ZONES all -- * * 0.0.0.0/0 0.0.0.0/0
0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate INVALID
87 13974 REJECT all -- * * 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 ACCEPT all -- * virbr0 0.0.0.0/0 192.168.122.0/24 ctstate RELATED,ESTABLISHED
0 0 ACCEPT all -- virbr0 * 192.168.122.0/24 0.0.0.0/0
0 0 ACCEPT all -- virbr0 virbr0 0.0.0.0/0 0.0.0.0/0
0 0 REJECT all -- * virbr0 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable
0 0 REJECT all -- virbr0 * 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable
0 0 ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
0 0 ACCEPT all -- lo * 0.0.0.0/0 0.0.0.0/0
0 0 FORWARD_direct all -- * * 0.0.0.0/0 0.0.0.0/0
0 0 FORWARD_IN_ZONES_SOURCE all -- * * 0.0.0.0/0 0.0.0.0/0
0 0 FORWARD_IN_ZONES all -- * * 0.0.0.0/0 0.0.0.0/0
0 0 FORWARD_OUT_ZONES_SOURCE all -- * * 0.0.0.0/0 0.0.0.0/0
0 0 FORWARD_OUT_ZONES all -- * * 0.0.0.0/0 0.0.0.0/0
0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate INVALID
0 0 REJECT all -- * * 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited
Chain OUTPUT (policy ACCEPT 666 packets, 73523 bytes)
pkts bytes target prot opt in out source destination
0 0 ACCEPT udp -- * virbr0 0.0.0.0/0 0.0.0.0/0 udp dpt:68
768 90999 OUTPUT_direct all -- * * 0.0.0.0/0 0.0.0.0/0
Chain FORWARD_IN_ZONES (1 references)
pkts bytes target prot opt in out source destination
0 0 FWDI_public all -- ens33 * 0.0.0.0/0 0.0.0.0/0 [goto]
0 0 FWDI_public all -- + * 0.0.0.0/0 0.0.0.0/0 [goto]
Chain FORWARD_IN_ZONES_SOURCE (1 references)
pkts bytes target prot opt in out source destination
Chain FORWARD_OUT_ZONES (1 references)
pkts bytes target prot opt in out source destination
0 0 FWDO_public all -- * ens33 0.0.0.0/0 0.0.0.0/0 [goto]
0 0 FWDO_public all -- * + 0.0.0.0/0 0.0.0.0/0 [goto]
Chain FORWARD_OUT_ZONES_SOURCE (1 references)
pkts bytes target prot opt in out source destination
Chain FORWARD_direct (1 references)
pkts bytes target prot opt in out source destination
Chain FWDI_public (2 references)
pkts bytes target prot opt in out source destination
0 0 FWDI_public_log all -- * * 0.0.0.0/0 0.0.0.0/0
0 0 FWDI_public_deny all -- * * 0.0.0.0/0 0.0.0.0/0
0 0 FWDI_public_allow all -- * * 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT icmp -- * * 0.0.0.0/0 0.0.0.0/0
Chain FWDI_public_allow (1 references)
pkts bytes target prot opt in out source destination
Chain FWDI_public_deny (1 references)
pkts bytes target prot opt in out source destination
Chain FWDI_public_log (1 references)
pkts bytes target prot opt in out source destination
Chain FWDO_public (2 references)
pkts bytes target prot opt in out source destination
0 0 FWDO_public_log all -- * * 0.0.0.0/0 0.0.0.0/0
0 0 FWDO_public_deny all -- * * 0.0.0.0/0 0.0.0.0/0
0 0 FWDO_public_allow all -- * * 0.0.0.0/0 0.0.0.0/0
Chain FWDO_public_allow (1 references)
pkts bytes target prot opt in out source destination
Chain FWDO_public_deny (1 references)
pkts bytes target prot opt in out source destination
Chain FWDO_public_log (1 references)
pkts bytes target prot opt in out source destination
Chain INPUT_ZONES (1 references)
pkts bytes target prot opt in out source destination
74 9875 IN_public all -- ens33 * 0.0.0.0/0 0.0.0.0/0 [goto]
9 1783 IN_public all -- + * 0.0.0.0/0 0.0.0.0/0 [goto]
Chain INPUT_ZONES_SOURCE (1 references)
pkts bytes target prot opt in out source destination
Chain INPUT_direct (1 references)
pkts bytes target prot opt in out source destination
Chain IN_public (2 references)
pkts bytes target prot opt in out source destination
95 14462 IN_public_log all -- * * 0.0.0.0/0 0.0.0.0/0
95 14462 IN_public_deny all -- * * 0.0.0.0/0 0.0.0.0/0
95 14462 IN_public_allow all -- * * 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT icmp -- * * 0.0.0.0/0 0.0.0.0/0
Chain IN_public_allow (1 references)
pkts bytes target prot opt in out source destination
8 488 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:22 ctstate NEW
Chain IN_public_deny (1 references)
pkts bytes target prot opt in out source destination
Chain IN_public_log (1 references)
pkts bytes target prot opt in out source destination
Chain OUTPUT_direct (1 references)
pkts bytes target prot opt in out source destination
[root@linux-02 ~]# systemctl stop firewalld.service
[root@linux-02 ~]#
## hold A The firewall of the machine also stopped ##
[root@linux-01 ~]# systemctl stop firewalld.service
[root@linux-01 ~]#
## Re execute the synchronization command, and it is found that there is still an error ##
[root@linux-02 ~]# rsync -av /root/rsync/ 192.168.241.88::rsync/tb89
@ERROR: access denied to rsync from UNKNOWN (192.168.241.89)
rsync error: error starting client-server protocol (code 5) at main.c(1648) [sender=3.1.2]
[root@linux-02 ~]#
## The reasons for the above error may be: https://www.jb51.net/article/31920.htm, this directory is not added after the configuration file is made. Add the next directory##
[root@linux-01 ~]# mkdir /tmp/rsync/
[root@linux-01 ~]# chmod 777 /tmp/rsync/
[root@linux-01 ~]#
## Prompt or error. The reason for the final solution is the error in the configuration file( hosts allow=192.168.241.88) This is wrong. It should be B Mechanical ip After the address is modified, you will be prompted to enter the password when using the transmission command, indicating that we have successfully configured it ##
[root@linux-02 ~]# rsync -av /root/rsync/ 192.168.241.88::rsync/tb89
@ERROR: access denied to rsync from UNKNOWN (192.168.241.89)
rsync error: error starting client-server protocol (code 5) at main.c(1648) [sender=3.1.2]
[root@linux-02 ~]# rsync -av /root/rsync/ 192.168.241.88::rsync/tb89
Password: In the configuration file, we have a file with a specified password. If this parameter and the above parameters are not specified, password authentication will not be used. We can comment the "auth users=test and secrets file=/etc/rsyncd.passwd" in the configuration file first to synchronize the file directly. If we don't comment it here, it is equivalent to specifying the user password for synchronization, which is relatively installed, User password profile format: user: password.
[root@linux-01 ~]# vim /etc/rsyncd.conf # /etc/rsyncd: configuration file for rsync daemon mode # See rsyncd.conf man page for more options. # configuration example: # uid = nobody # gid = nobody # use chroot = yes # max connections = 4 # pid file = /var/run/rsyncd.pid # exclude = lost+found/ # transfer logging = yes # timeout = 900 # ignore nonreadable = yes # dont compress = *.gz *.tgz *.zip *.z *.Z *.rpm *.deb *.bz2 # [ftp] # path = /home/ftp # comment = ftp export area port=873 log file=/var/log/rsync.log pid file=/var/run/rsyncd.pid address=192.168.241.88 [rsync] path=/tmp/rsync use chroot=true max connections=4 read only=no list=true uid=root gid=root #auth users=test / / comment out #secrets file=/etc/rsyncd.passwd / / comment out hosts allow=192.168.241.89 ## In machine B By executing the synchronization command on, you can see that the synchronized files have been synchronized ## [root@linux-02 ~]# rsync -av /root/rsync/ 192.168.241.88::rsync/tb89 sending incremental file list created directory /tb89 ./ pass.txt firewalld/ firewalld/firewalld.conf firewalld/lockdown-whitelist.xml firewalld/helpers/ firewalld/icmptypes/ firewalld/ipsets/ firewalld/services/ firewalld/zones/ firewalld/zones/public.xml firewalld/zones/public.xml.old sent 5,816 bytes received 174 bytes 570.48 bytes/sec total size is 5,226 speedup is 0.87 [root@linux-02 ~]#
Modify the list to false in the configuration file to hide the module name
[root@linux-02 ~]# rsync -av /root/rsync/ 192.168.241.88:: rsync [root@linux-02 ~]# ## modify list=false You can hide the module name ##
Set "– password file = / etc / rsync_passwd. TXT" on client B without entering password synchronization; This applies to scripts because it is where you need to deal with users.
## To create a password file, just enter server The password of the client can be saved ##
[root@linux-02 ~]# vim /etc/rsync_passwd.txt
[root@linux-02 ~]# chmod 600 /etc/rsync_passwd.txt
[root@linux-02 ~]# rsync -avP /root/rsync/ --password-file=/etc/rsync_passwd.txt 192.168.241.88::rsync/tb02
sending incremental file list
created directory /tb02
./
pass.txt
2,319 100% 0.00kB/s 0:00:00 (xfr#1, to-chk=10/12)
firewalld/
firewalld/firewalld.conf
2,006 100% 979.49kB/s 0:00:00 (xfr#2, to-chk=8/12)
firewalld/lockdown-whitelist.xml
271 100% 132.32kB/s 0:00:00 (xfr#3, to-chk=7/12)
firewalld/helpers/
firewalld/icmptypes/
firewalld/ipsets/
firewalld/services/
firewalld/zones/
firewalld/zones/public.xml
315 100% 153.81kB/s 0:00:00 (xfr#4, to-chk=1/12)
firewalld/zones/public.xml.old
315 100% 153.81kB/s 0:00:00 (xfr#5, to-chk=0/12)
sent 5,816 bytes received 174 bytes 3,993.33 bytes/sec
total size is 5,226 speedup is 0.875. System log
The log records all kinds of things that happen in the system every day. Seven is to aim at the system status, troubleshoot military faults, etc. you can use it to check the causes of errors. The main function of the log is audit and monitoring, as well as real-time monitoring of system status, monitoring and tracking intruders, etc
5.1 system log cutting
Daily system log / var/log/message; It is the core system log file, which contains the boot messages when the system is started and other status messages when the system is running. IO errors, network errors and other system errors will be recorded in this file. In addition, other information, such as a person's identity switching to root and the log of user-defined installed software (apache) will also be listed here.
[root@linux-01 ~]# ll /var/log/messages -rw-------. 1 root root 11680 4 June 20-17:20 /var/log/messages [root@linux-01 ~]# du -sh !$ du -sh /var/log/messages 12K /var/log/messages [root@linux-01 ~]# ls /var/log/messages* /var/log/messages /var/log/messages-20190303 /var/log/messages-20190312 /var/log/messages-20190420 [root@linux-01 ~]#
When viewing the log, you can see that the log is cut. What is the reason? The reason is that there is a logrotate service in the system, which will automatically cut logs to prevent the unlimited increase of a log file.
[root@linux-01 ~]# cat /etc/logrotate.conf
# see "man logrotate" for details
# rotate log files weekly
weekly //Cut once a week
# keep 4 weeks worth of backlogs
rotate 4 //4-week rotation
# create new (empty) log files after rotating old ones
create //Create a new
# use date as a suffix of the rotated file
dateext //Take it as the suffix
# uncomment this if you want your log files compressed
#compress / / whether compression is required. The compressed file format is tar.gz
# RPM packages drop log rotation information into this directory
include /etc/logrotate.d
# no packages own wtmp and btmp -- we'll rotate them here
/var/log/wtmp {
monthly
create 0664 root utmp
minsize 1M
rotate 1
}
/var/log/btmp {
missingok
monthly
create 0600 root utmp
rotate 1
}
# system-specific logs may be also be configured here. // System specific logs are configured here
[root@linux-01 ~]# ls /etc/logrotate.d/
bootlog chrony cups glusterfs iscsiuiolog libvirtd libvirtd.qemu numad ppp psacct samba sssd syslog wpa_supplicant yum- The syslog file will cut the cron, maillog, messages, secure and spool logs
- The messages log is determined by the syslogd service, so kill -HUP will reload the log
- There is also a script, shell command line. After cutting the log (removing it), change the name to generate a new log
- Linux system has a feature that when a service writes a file, it is not written according to the file name, but according to inode
[root@linux-01 ~]# cat /etc/logrotate.d/syslog
/var/log/cron
/var/log/maillog
/var/log/messages
/var/log/secure
/var/log/spooler
{
missingok
sharedscripts
postrotate
/bin/kill -HUP `cat /var/run/syslogd.pid 2> /dev/null` 2> /dev/null || true
endscript
}
[root@linux-01 ~]# 5.2 dmesg commands and logs
The dmesg command displays the ring buffer information of the linux kernel, from which we can obtain a large amount of system information at multiple operating levels, such as system architecture, cpu, mounted hardware, RAM and so on. When the computer starts, the system kernel (the core part of the operating system) will be loaded into memory. During the loading process, a lot of information will be displayed. In these information, we can see that the kernel detects hardware devices.
## This log is saved in memory. If the network card or hard disk device is damaged, it will be recorded in this log ## [root@linux-01 ~]# dmesg > 1.txt && tail -10f 1.txt [ 21.705910] virbr0: port 1(virbr0-nic) entered listening state [ 21.706034] IPv6: ADDRCONF(NETDEV_UP): virbr0: link is not ready [ 21.939927] virbr0: port 1(virbr0-nic) entered disabled state [ 4475.234487] Ebtables v2.0 unregistered [ 6132.941872] hrtimer: interrupt took 30718844 ns [ 6138.527461] sched: RT throttling activated [ 7236.889644] e1000: ens33 NIC Link is Down [ 7240.894546] e1000: ens33 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None [11215.055445] e1000: ens33 NIC Link is Down [11228.655532] e1000: ens33 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None ## dmesg -c Clear current log in memory ## [root@linux-01 ~]# Dmesg - C | head / / no log is generated [root@linux-01 ~]#
Check the / var/log/dmesg log, which contains kernel ring buffer information. When the system starts, a lot of hardware related information will be displayed on the screen.
[root@linux-01 ~]# tail -20f /var/log/dmesg [ 12.083499] XFS (sda2): Ending clean mount [ 12.222580] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 12.712893] cryptd: max_cpu_qlen set to 1000 [ 12.789277] AVX version of gcm_enc/dec engaged. [ 12.789281] AES CTR mode by8 optimization enabled [ 12.903329] alg: No test for __gcm-aes-aesni (__driver-gcm-aes-aesni) [ 12.923177] alg: No test for __generic-gcm-aes-aesni (__driver-generic-gcm-aes-aesni) [ 13.201356] ppdev: user-space parallel port driver [ 13.357443] floppy0: no floppy controllers found [ 13.357506] work still pending [ 13.831001] device-mapper: uevent: version 1.0.3 [ 13.831141] device-mapper: ioctl: 4.37.1-ioctl (2018-04-03) initialised: dm-devel@redhat.com [ 14.077935] type=1305 audit(1555725356.320:3): audit_pid=6161 old=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:auditd_t:s0 res=1 [ 14.131453] RPC: Registered named UNIX socket transport module. [ 14.131457] RPC: Registered udp transport module. [ 14.131458] RPC: Registered tcp transport module. [ 14.131460] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 14.966374] Bluetooth: BNEP (Ethernet Emulation) ver 1.3 [ 14.966378] Bluetooth: BNEP filters: protocol multicast [ 14.966385] Bluetooth: BNEP socket layer initialized
5.3 last command
- The last command is the called / var/log/wtmp log
- The log content includes: user, login terminal, source IP, time and login duration
- /var/log/wtmp log is a binary file and cannot be viewed directly. You can only view it with the last command
[root@linux-01 ~]# last | tail -20f reboot system boot 3.10.0-957.el7.x Tue Mar 12 20:23 - 22:47 (1+02:23) root pts/0 192.168.241.1 Sun Mar 3 11:40 - crash (9+08:42) root pts/0 192.168.241.1 Fri Mar 1 21:11 - 23:24 (02:13) reboot system boot 3.10.0-957.el7.x Fri Mar 1 21:11 - 22:47 (12+01:36) root pts/1 192.168.241.1 Thu Feb 28 23:32 - crash (21:38) whdong00 :0 :0 Thu Feb 28 21:25 - crash (23:45) whdong00 pts/1 :0 Thu Feb 28 21:22 - 21:22 (00:00) root pts/0 192.168.241.1 Thu Feb 28 21:21 - crash (23:49) whdong00 :0 :0 Thu Feb 28 21:21 - 21:25 (00:04) root tty1 Thu Feb 28 21:17 - 21:17 (00:00) reboot system boot 3.10.0-957.el7.x Thu Feb 28 21:16 - 22:47 (13+01:30) root pts/0 192.168.241.1 Thu Feb 28 21:01 - crash (00:15) root tty1 Thu Feb 28 20:52 - 21:01 (00:09) reboot system boot 3.10.0-957.el7.x Thu Feb 28 20:52 - 22:47 (13+01:54) root tty1 Thu Feb 28 20:46 - crash (00:05) reboot system boot 3.10.0-957.el7.x Thu Feb 28 20:46 - 22:47 (13+02:00) root tty1 Fri Mar 1 04:33 - 20:46 (-7:-47) reboot system boot 3.10.0-957.el7.x Fri Mar 1 04:33 - 20:46 (-7:-46) wtmp begins Fri Mar 1 04:33:09 2019 [root@linux-01 ~]#
5.4 use the lastb command to view users who have failed to log in
- The lastb command is the called / var/log/btmp log
- The log content includes: user, login terminal, login IP, time and login duration
- /var/log/btmp log is also a binary file, which cannot be viewed directly. You can only view it with the lastb command
[root@linux-01 ~]# lastb root ssh:notty 192.168.241.89 Sat Apr 20 17:58 - 17:58 (00:00) root ssh:notty 192.168.241.89 Sat Apr 20 17:58 - 17:58 (00:00) btmp begins Sat Apr 20 17:58:48 2019 [root@linux-01 ~]#
5.5 System Safety log
/var/log/secure, for example, log in to the operating system. A log will be recorded here if the verification is successful, and a log will be recorded if it fails
[root@linux-01 ~]# tail -20f /var/log/secure Apr 20 11:10:27 linux-01 polkitd[6209]: Registered Authentication Agent for unix-process:9129:448474 (system bus name :1.85 [/usr/bin/pkttyagent --notify-fd 5 --fallback], object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale zh_CN.UTF-8) Apr 20 11:10:28 linux-01 polkitd[6209]: Unregistered Authentication Agent for unix-process:9129:448474 (system bus name :1.85, object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale zh_CN.UTF-8) (disconnected from bus) Apr 20 11:56:29 linux-01 sshd[7516]: pam_unix(sshd:session): session closed for user root Apr 20 12:40:16 linux-01 sshd[9750]: Accepted password for root from 192.168.241.1 port 55096 ssh2 Apr 20 12:40:17 linux-01 sshd[9750]: pam_unix(sshd:session): session opened for user root by (uid=0) Apr 20 16:37:34 linux-01 sshd[10429]: Accepted password for root from 192.168.241.1 port 55185 ssh2 Apr 20 16:37:34 linux-01 sshd[10429]: pam_unix(sshd:session): session opened for user root by (uid=0) Apr 20 17:30:20 linux-01 sshd[8196]: pam_unix(sshd:session): session closed for user root Apr 20 17:58:46 linux-01 unix_chkpwd[11337]: password check failed for user (root) Apr 20 17:58:46 linux-01 sshd[11335]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=192.168.241.89 user=root Apr 20 17:58:46 linux-01 sshd[11335]: pam_succeed_if(sshd:auth): requirement "uid >= 1000" not met by user "root" Apr 20 17:58:48 linux-01 sshd[11335]: Failed password for root from 192.168.241.89 port 33498 ssh2 Apr 20 17:58:50 linux-01 unix_chkpwd[11338]: password check failed for user (root) Apr 20 17:58:50 linux-01 sshd[11335]: pam_succeed_if(sshd:auth): requirement "uid >= 1000" not met by user "root" Apr 20 17:58:52 linux-01 sshd[11335]: Failed password for root from 192.168.241.89 port 33498 ssh2 Apr 20 17:58:52 linux-01 sshd[11335]: Connection closed by 192.168.241.89 port 33498 [preauth] Apr 20 17:58:52 linux-01 sshd[11335]: PAM 1 more authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=192.168.241.89 user=root
6. screen tool - virtual terminal
When we need to execute a task for a long time, we can't exit from the remote terminal. What is the way to continue the task without disconnecting the task? We can use "nohup command > > 1. Log &" to place the task in the background. When we want to view the information on the screen, we can't see it, This allows you to use the screen tool to start a new virtual terminal. The system default screen is not installed. We need to install it.
[root@linux-01 ~]# yum install -y screen Plug in loaded: fastestmirror, langpacks Loading mirror speeds from cached hostfile * base: centos.ustc.edu.cn * extras: mirrors.cn99.com * updates: mirrors.cn99.com base | 3.6 kB 00:00:00 extras | 3.4 kB 00:00:00 updates | 3.4 kB 00:00:00 Resolving dependencies --> Checking transactions ---> software package screen.x86_64.0.4.1.0-0.25.20120314git3c2946.el7 Will be installed --> Resolve dependency complete Dependency resolution ================================================================================================================================================== Package framework edition source size ================================================================================================================================================== Installing: screen x86_64 4.1.0-0.25.20120314git3c2946.el7 base 552 k Transaction summary ================================================================================================================================================== Install 1 package Total downloads: 552 k Installation size: 914 k Downloading packages: screen-4.1.0-0.25.20120314git3c2946.el7.x86_64.rpm | 552 kB 00:00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : screen-4.1.0-0.25.20120314git3c2946.el7.x86_64 1/1 Verification in progress : screen-4.1.0-0.25.20120314git3c2946.el7.x86_64 1/1 already installed: screen.x86_64 0:4.1.0-0.25.20120314git3c2946.el7 complete!
## Run command screen You can start a virtual terminal ## [root@linux-01 ~]# screen ## Run one in the virtual terminal vmstat 1 ，use ctrl+a+d You can exit the virtual terminal## [root@linux-01 ~]# screen [detached from 11624.pts-2.linux-01] ## How do I get back to this terminal ## [root@linux-01 ~]# Screen - LS / / view the number of virtual terminals There is a screen on: 11624.pts-2.linux-01 (Detached) 1 Socket in /var/run/screen/S-root. [root@linux-01 ~]# Screen - r 11624 / / use the - r option to add the ID number of the terminal to return to the terminal ## If there are multiple screen For a long time, I don't know what I need. What should I do? I can create it in screen Add it when you need it-S Option to specify a name ## [root@linux-01 ~]# screen -S xihaji [detached from 11755.xihaji] [root@linux-01 ~]# screen -ls There are screens on: 11755.xihaji (Detached) 11691.pts-2.linux-01 (Detached) 11624.pts-2.linux-01 (Detached) 3 Sockets in /var/run/screen/S-root. [root@linux-01 ~]# screen -r 11755 / / you can easily find your own screen
After class summary
1. Usage scenario of Rsync
2. Log cutter logrotate https://linux.cn/article-4126-1.html
3.xargs https://blog.csdn.net/gb4215287/article/details/78037520