Complete collection of linux commands

Posted by jemgames on Wed, 05 Jan 2022 04:37:47 +0100

Play with Linux operating system

Note: the explanation of Linux commands in this article is based on the Linux distribution named CentOS. I use Alibaba cloud server, and the system version is CentOS Linux release 7.6.1810. Different Linux distributions have some differences in Shell commands and utilities, but these differences are small.

History of operating system

The computer system with only hardware and no software is called "bare metal". It is difficult for us to use "bare metal" to complete the daily work of the computer (such as storage and operation), so we must use specific software to control the work of the hardware. The software closest to the computer hardware is the system software, of which the most important is the "operating system". "Operating system" is a collection of programs that control and manage the whole computer hardware and software resources, realize resource allocation and task allocation, and provide interfaces and environments for system users and other software.

No operating system (manual operation)

When there was no operating system at the beginning of the birth of the computer, people first loaded the program paper tape (or card) into the computer, then started the input machine to send the program into the computer, and then started the program through the console switch. When the program is executed, the printer outputs the calculation results, and the user removes and takes away the paper tape (or card). The second user gets on the computer and repeats the same steps. In the whole process, the user monopolizes the machine, the CPU waits for manual operation, and the resource utilization is very low.

Batch processing system

First, start a supervision program on the computer. Under the control of the supervision program, the computer can automatically and batch process the jobs of one or more users. After a batch of jobs are completed, the supervision level reads the jobs from the input machine and stores them in the tape drive. Repeat the task as above. The supervision program continuously processes each job, realizes the automatic transfer of jobs, reduces the establishment time and manual operation time of jobs, and improves the utilization of computer resources. Batch processing system can be divided into single channel batch processing system, multi-channel batch processing system, online batch processing system and offline batch processing system.

Time sharing system and real-time system

The time-sharing system divides the running time of the processor into very short time slices, and allocates the processor to each online job in turn. If a job cannot complete its calculation within the time slice allocated to it, the job will be temporarily interrupted and the processor will be given to another job for use, waiting for the next round of scheduling before continuing its operation. Due to the fast speed of the computer and the fast rotation of the job, each user feels that he has monopolized a computer. Each user can send various operation and control commands to the system through his own terminal to complete the operation of the job under the condition of full human-computer interaction. In order to solve the problem that the time-sharing system can not respond to user instructions in time, there is a real-time system that can complete event processing in a strict time range and respond to random external events in time.

General operating system

1960s: IBM's System/360 series machines have a unified operating system OS/360.
1965: Bell Labs of at & T joined the cooperation program of GE and MIT and began to develop MULTICS.
1969: the MULTICS project failed. Ken Tompson was idle at home. In order to play the "Space Travel" game, he developed Unics on the obsolete PDP-7 with assembly language.

Note: it is hard to imagine that such a great system as Unix was developed by an idle programmer (the key is that his wife returned to his mother's house with children) to play games on an obsolete device.
1970 ~ 1971: Ken Tompson and Dennis Ritchie rewritten Unics on PDP-11 in B language and renamed it Unix at the suggestion of Brian Kernighan.
1972 ~ 1973: Dennis Ritchie invented C language to replace B language with poor portability, and started rewriting Unix with C language.
1974: Unix launched the landmark fifth edition, which was almost completely implemented in C language.
1979: from the 7th edition of Unix, at & T issued new terms of use to privatize Unix.
1987: Professor Andrew S. Tanenbaum decided to develop a Unix compatible operating system without using any at & T source code to avoid copyright disputes in order to explain the details of the operation of the operating system to students in class. The system was named Minix.
1991: Linus Torvalds tried to do some development work on Minix while studying at the University of Helsinki, Finland, but because Minix is only an operating system for teaching purposes and its function is not powerful, Linus wrote a disk driver and file system to facilitate reading, writing and downloading files in the school's newsgroup and mail system, These things form the rudiment of the Linux system kernel.

The following figure is a map of the Unix operating system family.

Linux overview

Linux is a general operating system. An operating system should be responsible for task scheduling, memory allocation, processing peripheral I/O and other operations. The operating system is usually composed of two parts: the kernel (running other programs and managing the core programs of hardware devices such as disk and printer) and the system program (device driver, underlying library, shell, service program, etc.).

The Linux kernel was developed by Finnish Linus Torvalds and released in September 1991. As a product of the Internet era, Linux operating system is jointly developed by many developers all over the world. It is a free operating system (note that freedom and free are not the same concept. If you want to understand the difference between them, you can click here ).

Linux system advantages

General operating system, not bound to specific hardware.
Written in C language, it has strong portability and kernel programming interface.
Support multi-user and multi task, and support secure hierarchical file system.
A large number of utilities, perfect network functions and powerful supporting documents.
Reliable security and good stability, more friendly to developers.

Linux system distribution

Basic command

Commands on Linux systems are usually in the following format:

Command name [Named parameters] [Command object]

Get login information - w / who / last/ lastb.

[root ~]# w
 23:31:16 up 12:16,  2 users,  load average: 0.00, 0.01, 0.05
USER     TTY      FROM             LOGIN@   IDLE   JCPU   PCPU WHAT
root     pts/0    182.139.66.250   23:03    4.00s  0.02s  0.00s w
jackfrue pts/1    182.139.66.250   23:26    3:56   0.00s  0.00s -bash
[root ~]# who
root     pts/0        2018-04-12 23:03 (182.139.66.250)
jackfrued pts/1        2018-04-12 23:26 (182.139.66.250)
[root ~]# who am i
root     pts/0        2018-04-12 23:03 (182.139.66.250)
[root ~]# who mom likes
root     pts/0        2018-04-12 23:03 (182.139.66.250)
[root ~]# last
root     pts/0        117.136.63.184   Sun May 26 18:57   still logged in   
reboot   system boot  3.10.0-957.10.1. Mon May 27 02:52 - 19:10  (-7:-42)   
root     pts/4        117.136.63.184   Sun May 26 18:51 - crash  (08:01)    
root     pts/4        117.136.63.184   Sun May 26 18:49 - 18:49  (00:00)    
root     pts/3        117.136.63.183   Sun May 26 18:35 - crash  (08:17)    
root     pts/2        117.136.63.183   Sun May 26 18:34 - crash  (08:17)    
root     pts/0        117.136.63.183   Sun May 26 18:10 - crash  (08:42)

View your own Shell - ps.

The interface between the user and the Shell is called the "Shell interface", that is, the interface between the user and the Shell. At present, the default Shell of many Linux systems is bash (Bourne Again SHell), because it can use the tab key to complete commands and paths, save historical commands, easily configure environment variables and execute batch operations.
```
[root ~]# ps
  PID TTY          TIME CMD
 3531 pts/0    00:00:00 bash
 3553 pts/0    00:00:00 ps
```

View the description and location of the command - which / which / whereis.

[root ~]# whatis ps
ps (1)        - report a snapshot of the current processes.
[root ~]# whatis python
python (1)    - an interpreted, interactive, object-oriented programming language
[root ~]# whereis ps
ps: /usr/bin/ps /usr/share/man/man1/ps.1.gz
[root ~]# whereis python
python: /usr/bin/python /usr/bin/python2.7 /usr/lib/python2.7 /usr/lib64/python2.7 /etc/python /usr/include/python2.7 /usr/share/man/man1/python.1.gz
[root ~]# which ps
/usr/bin/ps
[root ~]# which python
/usr/bin/python

Clear what is displayed on the screen - clear.

View help documentation - man / info / --help / apropos.

[root@izwz97tbgo9lkabnat2lo8z ~]# ps --help
Usage:
 ps [options]
 Try 'ps --help <simple|list|output|threads|misc|all>'
  or 'ps --help <s|l|o|t|m|a>'
 for additional help text.
For more details see ps(1).
[root@izwz97tbgo9lkabnat2lo8z ~]# man ps
PS(1)                                User Commands                                PS(1)
NAME
       ps - report a snapshot of the current processes.
SYNOPSIS
       ps [options]
DESCRIPTION
...

View the system and hostname - uname / hostname.
```
[root@izwz97tbgo9lkabnat2lo8z ~]# uname
Linux
[root@izwz97tbgo9lkabnat2lo8z ~]# hostname
izwz97tbgo9lkabnat2lo8z
[root@iZwz97tbgo9lkabnat2lo8Z ~]# cat /etc/centos-release
CentOS Linux release 7.6.1810 (Core)
```
Note: cat is the command to connect file content and print to standard output, which will be described later/ etc is a very important directory on Linux system. It saves a lot of configuration files; CentOS release is a file in this directory. Because the Linux distribution version I use is CentOS 7.6, there will be such a file here.

Time and date - date / cal.

[root@iZwz97tbgo9lkabnat2lo8Z ~]# date
Wed Jun 20 12:53:19 CST 2018
[root@iZwz97tbgo9lkabnat2lo8Z ~]# cal
      June 2018
Su Mo Tu We Th Fr Sa
                1  2
 3  4  5  6  7  8  9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
[root@iZwz97tbgo9lkabnat2lo8Z ~]# cal 5 2017
      May 2017
Su Mo Tu We Th Fr Sa
    1  2  3  4  5  6
 7  8  9 10 11 12 13
14 15 16 17 18 19 20
21 22 23 24 25 26 27
28 29 30 31

Restart and shutdown - reboot / shutdown.

[root ~]# shutdown -h +5
Shutdown scheduled for Sun 2019-05-26 19:34:27 CST, use 'shutdown -c' to cancel.
[root ~]# 
Broadcast message from root (Sun 2019-05-26 19:29:27 CST):

The system is going down for power-off at Sun 2019-05-26 19:34:27 CST!
[root ~]# shutdown -c

Broadcast message from root (Sun 2019-05-26 19:30:22 CST):

The system shutdown has been cancelled at Sun 2019-05-26 19:31:22 CST!
[root ~]# shutdown -r 23:58
Shutdown scheduled for Sun 2019-05-26 23:58:00 CST, use 'shutdown -c' to cancel.
[root ~]# shutdown -c

Broadcast message from root (Sun 2019-05-26 19:31:06 CST):

The system shutdown has been cancelled at Sun 2019-05-26 19:32:06 CST!

Note: when executing the shutdown command, a warning will be issued to the user logging in to the system. You can replace the default warning message by following the command with the warning message, or you can use now after the - h parameter to indicate immediate shutdown.

Exit login - exit / logout.
View history command - history.

[root@iZwz97tbgo9lkabnat2lo8Z ~]# history
...
452  ls
453  cd Python-3.6.5/
454  clear
455  history
[root@iZwz97tbgo9lkabnat2lo8Z ~]# !454

Note: after viewing the history command, you can use! Re execute the command according to the historical command number; The history command can be cleared through history -c.

Utilities

File and folder operations

Create / delete empty directory - mkdir / rmdir.

[root ~]# mkdir abc
[root ~]# mkdir -p xyz/abc
[root ~]# rmdir abc

Create / delete file - touch / rm.
```
[root ~]# touch readme.txt
[root ~]# touch error.txt
[root ~]# rm error.txt
rm: remove regular empty file 'error.txt'? y
[root ~]# rm -rf xyz
```
- The touch command is used to create a blank file or modify the file time. In Linux system, a file has three times:
  - Time of content change - mtime.
  - Time to change permissions - ctime.
  - Last access time - atime.
- Several important parameters of rm:
  - -i: Interactive deletion, each deleted item will be asked.
  - -r: Delete the directory and recursively delete the files and directories in the directory.
  - -f: Forced deletion, ignoring nonexistent files without any prompt.
Switch and view the current working directory - cd / pwd.

Note: the cd command can be followed by a relative path (referring to the current path) or an absolute path (starting with /) to switch to the specified directory, or cd To return to the previous directory. Please think about what parameters should be added to the cd command if you want to return to the upper directory?
View directory contents - ls.
- -l: View files and directories in long format.
- -a: Displays files and directories that begin with a dot (hidden files).
- -R: Recursively expand the directory (continue to list the files and directories under the directory).
- -d: Only the directory is listed, and nothing else is listed.
- -S / -t: sort by size / time.

View the contents of the file - cat / tac / head / tail / more / less / rev / od.

[root ~]# wget http://www.sohu.com/ -O sohu.html
--2018-06-20 18:42:34--  http://www.sohu.com/
Resolving www.sohu.com (www.sohu.com)... 14.18.240.6
Connecting to www.sohu.com (www.sohu.com)|14.18.240.6|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 212527 (208K) [text/html]
Saving to: 'sohu.html'
100%[==================================================>] 212,527     --.-K/s   in 0.03s
2018-06-20 18:42:34 (7.48 MB/s) - 'sohu.html' saved [212527/212527]
[root ~]# cat sohu.html
...
[root ~]# head -10 sohu.html
<!DOCTYPE html>
<html>
<head>
<title>Sohu</title>
<meta name="Keywords" content="Sohu,portal site,new media,Network media,Journalism,Finance and Economics,Sports,entertainment,fashion,automobile,house property,science and technology,picture,forum,micro-blog,Blog,video,film,TV play"/>
<meta name="Description" content="Sohu provides users with 24-hour up-to-date information, search, e-mail and other network services. The content includes global hot events, breaking news, current affairs comments, popular film and television dramas, sports events, industry trends, life service information, as well as interactive spaces such as forums, blogs, microblogs and my Sohu." />
<meta name="shenma-site-verification" content="1237e4d02a3d8d73e96cbd97b699e9c3_1504254750">
<meta charset="utf-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=Edge,chrome=1"/>
[root ~]# tail -2 sohu.html
</body>
</html>
[root ~]# less sohu.html
...
[root ~]# cat -n sohu.html | more
...

Note: a command named wget is used above. It is a network downloader program that can download resources from the specified URL.

Copy / move files - cp / mv.

[root ~]# mkdir backup
[root ~]# cp sohu.html backup/
[root ~]# cd backup
[root backup]# ls
sohu.html
[root backup]# mv sohu.html sohu_index.html
[root backup]# ls
sohu_index.html

File rename - Rename.

[root@iZwz97tbgo9lkabnat2lo8Z ~]# rename .htm .html *.htm

Find files and find content - find / grep.

[root@iZwz97tbgo9lkabnat2lo8Z ~]# find / -name "*.html"
/root/sohu.html
/root/backup/sohu_index.html
[root@izwz97tbgo9lkabnat2lo8z ~]# find . -atime 7 -type f -print
[root@izwz97tbgo9lkabnat2lo8z ~]# find . -type f -size +2k
[root@izwz97tbgo9lkabnat2lo8z ~]# find . -type f -name "*.swp" -delete
[root@iZwz97tbgo9lkabnat2lo8Z ~]# grep "<script>" sohu.html -n
20:<script>
[root@iZwz97tbgo9lkabnat2lo8Z ~]# grep -E \<\/?script.*\> sohu.html -n
20:<script>
22:</script>
24:<script src="//statics.itc.cn/web/v3/static/js/es5-shim-08e41cfc3e.min.js"></script>
25:<script src="//statics.itc.cn/web/v3/static/js/es5-sham-1d5fa1124b.min.js"></script>
26:<script src="//statics.itc.cn/web/v3/static/js/html5shiv-21fc8c2ba6.js"></script>
29:<script type="text/javascript">
52:</script>
...

Note: grep can use regular expressions when searching strings. If you need to use regular expressions, you can use grep -E or egrep directly.

Create links and view links - ln / readlink.
```
[root@iZwz97tbgo9lkabnat2lo8Z ~]# ls -l sohu.html
-rw-r--r-- 1 root root 212131 Jun 20 19:15 sohu.html
[root@iZwz97tbgo9lkabnat2lo8Z ~]# ln /root/sohu.html /root/backup/sohu_backup
[root@iZwz97tbgo9lkabnat2lo8Z ~]# ls -l sohu.html
-rw-r--r-- 2 root root 212131 Jun 20 19:15 sohu.html
[root@iZwz97tbgo9lkabnat2lo8Z ~]# ln /root/sohu.html /root/backup/sohu_backup2
[root@iZwz97tbgo9lkabnat2lo8Z ~]# ls -l sohu.html
-rw-r--r-- 3 root root 212131 Jun 20 19:15 sohu.html
[root@iZwz97tbgo9lkabnat2lo8Z ~]# ln -s /etc/centos-release sysinfo
[root@iZwz97tbgo9lkabnat2lo8Z ~]# ls -l sysinfo
lrwxrwxrwx 1 root root 19 Jun 20 19:21 sysinfo -> /etc/centos-release
[root@iZwz97tbgo9lkabnat2lo8Z ~]# cat sysinfo
CentOS Linux release 7.4.1708 (Core)
[root@iZwz97tbgo9lkabnat2lo8Z ~]# cat /etc/centos-release
CentOS Linux release 7.4.1708 (Core)
```
Note: links can be divided into hard links and soft links (symbolic links). A hard link can be regarded as a pointer to file data, just like the reference count of objects in Python. Each time a hard link is added, the number of links corresponding to the file increases by 1. Only when the number of links of the file is 0, the storage space corresponding to the file may be overwritten by other files. In fact, when we delete a file, we do not delete the data on the hard disk. What we delete is only a pointer, or a usage record of the data. Therefore, when "crushing" a file, software like "file shredder" will not only delete the file pointer, but also fill in the data in the corresponding storage area of the file to ensure that the file can no longer be recovered. The soft link is similar to the shortcut under Windows system. When the file linked by the soft link is deleted, the soft link will become invalid.
Compress / decompress and archive / de Archive - gzip / gunzip / xz.

[root@iZwz97tbgo9lkabnat2lo8Z ~]# wget http://download.redis.io/releases/redis-4.0.10.tar.gz
--2018-06-20 19:29:59--  http://download.redis.io/releases/redis-4.0.10.tar.gz
Resolving download.redis.io (download.redis.io)... 109.74.203.151
Connecting to download.redis.io (download.redis.io)|109.74.203.151|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1738465 (1.7M) [application/x-gzip]
Saving to: 'redis-4.0.10.tar.gz'
100%[==================================================>] 1,738,465   70.1KB/s   in 74s
2018-06-20 19:31:14 (22.9 KB/s) - 'redis-4.0.10.tar.gz' saved [1738465/1738465]
[root@iZwz97tbgo9lkabnat2lo8Z ~]# ls redis*
redis-4.0.10.tar.gz
[root@iZwz97tbgo9lkabnat2lo8Z ~]# gunzip redis-4.0.10.tar.gz
[root@iZwz97tbgo9lkabnat2lo8Z ~]# ls redis*
redis-4.0.10.tar

Archiving and de archiving - tar.

[root@iZwz97tbgo9lkabnat2lo8Z ~]# tar -xvf redis-4.0.10.tar
redis-4.0.10/
redis-4.0.10/.gitignore
redis-4.0.10/00-RELEASENOTES
redis-4.0.10/BUGS
redis-4.0.10/CONTRIBUTING
redis-4.0.10/COPYING
redis-4.0.10/INSTALL
redis-4.0.10/MANIFESTO
redis-4.0.10/Makefile
redis-4.0.10/README.md
redis-4.0.10/deps/
redis-4.0.10/deps/Makefile
redis-4.0.10/deps/README.md
...

Note: the tar command is used for archiving (also known as create Archive) and de archiving. Generally, three parameters - cvf are required to create an archive, where c represents create, v represents verbose, and f represents the file specified for archiving; The - xvf parameter needs to be added to de archive, where x represents extract, and the other two parameters are the same as creating archive.

Convert standard input to command line arguments - xargs.

The following command will find the html files in the current path, and then pass these files as parameters to the rm command through xargs to find and delete files.

[root@iZwz97tbgo9lkabnat2lo8Z ~]# find . -type f -name "*.html" | xargs rm -f

The following command changes multiple lines in the a.txt file into one line and outputs it to the b.txt file, where < means to read the input from a.txt and > means to output the execution result of the command to b.txt.

[root@iZwz97tbgo9lkabnat2lo8Z ~]# xargs < a.txt > b.txt

Note: as demonstrated above, this command is often used in pipeline (a way to realize inter process communication) and redirection (reassigning the location of input and output). Pipeline operation and input and output redirection will be discussed later.

Displays the file or directory - basename / dirname.
Other relevant tools.

Sort - sort content
uniq - remove adjacent duplicates
tr - replace specified content with new content
Cut / paste - cut / paste content
Split - split file
File - determine file type
wc - count the number of lines, words and bytes of the file
iconv - encoding conversion

[root ~]# cat foo.txt
grape
apple
pitaya
[root ~]# cat bar.txt
100
200
300
400
[root ~]# paste foo.txt bar.txt
grape   100
apple   200
pitaya  300
        400
[root ~]# paste foo.txt bar.txt > hello.txt
[root ~]# cut -b 4-8 hello.txt
pe      10
le      20
aya     3
0
[root ~]# cat hello.txt | tr '\t' ','
grape,100
apple,200
pitaya,300
,400
[root ~]# split -l 100 sohu.html hello
[root ~]# wget https://www.baidu.com/img/bd_logo1.png
[root ~]# file bd_logo1.png
bd_logo1.png: PNG image data, 540 x 258, 8-bit colormap, non-interlaced
[root ~]# wc sohu.html
  2979   6355 212527 sohu.html
[root ~]# wc -l sohu.html
2979 sohu.html
[root ~]# wget http://www.qq.com -O qq.html
[root ~]# iconv -f gb2312 -t utf-8 qq.html

Pipes and Redirection

Use of pipes - |.

Example: find the number of files in the current directory.
```
[root ~]# find ./ | wc -l
6152
```
Example: list the files and folders under the current path and add a number to each item.
```
[root ~]# ls | cat -n
     1  dump.rdb
     2  mongodb-3.6.5
     3  Python-3.6.5
     4  redis-3.2.11
     5  redis.conf
```
Example: find record The total number of records that contain AAA but not BBB in the log
```
[root ~]# cat record.log | grep AAA | grep -v BBB | wc -l
```

Output redirection and error redirection - > / > > / 2 >.

[root ~]# cat readme.txt
banana
apple
grape
apple
grape
watermelon
pear
pitaya
[root ~]# cat readme.txt | sort | uniq > result.txt
[root ~]# cat result.txt
apple
banana
grape
pear
pitaya
watermelon

Enter redirection - <.

[root ~]# echo 'hello, world!' > hello.txt
[root ~]# wall < hello.txt
[root ~]#
Broadcast message from root (Wed Jun 20 19:43:05 2018):
hello, world!
[root ~]# echo 'I will show you some code.' >> hello.txt
[root ~]# wall < hello.txt
[root ~]#
Broadcast message from root (Wed Jun 20 19:43:55 2018):
hello, world!
I will show you some code.

Multiple orientation - tee.

In addition to displaying the result of ls command on the terminal, the following command will also be appended to ls Txt file.
```
[root ~]# ls | tee -a ls.txt
```

alias

[root ~]# alias ll='ls -l'
[root ~]# alias frm='rm -rf'
[root ~]# ll
...
drwxr-xr-x  2 root       root   4096 Jun 20 12:52 abc
...
[root ~]# frm abc

unalias

[root ~]# unalias frm
[root ~]# frm sohu.html
-bash: frm: command not found

text processing

Character stream editor - sed.

sed is a tool for manipulating, filtering, and transforming text content. Suppose there is a named fruit Txt file, as shown below.
```
[root ~]# cat -n fruit.txt 
     1  banana
     2  grape
     3  apple
     4  watermelon
     5  orange
```
Next, we add a pitaya after line 2.
```
[root ~]# sed '2a pitaya' fruit.txt 
banana
grape
pitaya
apple
watermelon
orange
```
Note: the command just now, like many commands we talked about before, does not change fruit Txt file, but output the content added with a new line to the terminal, if you want to save it to fruit.txt Txt, you can use the output redirection operation.

Insert a waxberry before line 2.
```
[root ~]# sed '2i waxberry' fruit.txt
banana
waxberry
grape
apple
watermelon
orange
```
Delete line 3.
```
[root ~]# sed '3d' fruit.txt
banana
grape
watermelon
orange
```
Delete lines 2 through 4.
```
[root ~]# sed '2,4d' fruit.txt
banana
orange
```
Replace the character a in the text with @.
```
[root ~]# sed 's#a#@#' fruit.txt 
b@nana
gr@pe
@pple
w@termelon
or@nge
```
Replace the character a in the text with @, using global mode.
```
[root ~]# sed 's#a#@#g' fruit.txt 
b@n@n@
gr@pe
@pple
w@termelon
or@nge
```
Pattern matching and processing language - awk.

awk is a programming language and the most powerful tool for processing text in Linux system. One of its authors and current maintainers is Brian Kernighan (ken and dmr's closest partner) mentioned earlier. Through this command, we can extract the specified column from the text, use regular expression to extract the content we want from the text, display the specified row, and perform statistics and operations. In short, it is very powerful.

Suppose there is a named fruit 2 Txt file, as shown below.
```
[root ~]# cat fruit2.txt 
1       banana      120
2       grape       500
3       apple       1230
4       watermelon  80
5       orange      400
```
Line 3 of the file is displayed.
```
[root ~]# awk 'NR==3' fruit2.txt 
3       apple       1230
```
Displays column 2 of the file.
```
[root ~]# awk '{print $2}' fruit2.txt 
banana
grape
apple
watermelon
orange
```
Displays the last column of the file.
```
[root ~]# awk '{print $NF}' fruit2.txt 
120
500
1230
80
400
```
Output lines with the end number greater than or equal to 300.
```
[root ~]# awk '{if($3 >= 300) {print $0}}' fruit2.txt 
2       grape       500
3       apple       1230
5       orange      400
```
What is shown above is only the tip of the iceberg of the awk command. More content is left for readers to explore in practice.

user management

Create and delete users - useradd / userdel.
```
[root home]# useradd hellokitty
[root home]# userdel hellokitty
```
- -d - specify the user home directory for the user when creating the user
- -g - specify the user group to which the user belongs when creating the user
Create and delete user groups - groupadd / groupdel.

Note: user groups are mainly used to facilitate the management of all users in a group.
Change password - passwd.
```
[root ~]# passwd hellokitty
New password: 
Retype new password: 
passwd: all authentication tokens updated successfully.
```
Note: the input password and confirmation password are not echoed, and must be completed at one go (the backspace key cannot be used). The password and confirmation password need to be consistent. If there is no object specified for the command when using the passwd command, it indicates that you want to modify the password of the current user. If you want to batch modify user passwords, you can use the chpasswd command.
- -l / -u - lock / unlock user.
- -d - clear user password.
- -e - set the password to expire immediately, and the user will be forced to change the password when logging in.
- -i - set the number of days after the password expires to disable the user.
View and modify password validity - chage.

After setting up Hello Kitty user for 100 days, you must change the password, notify the user 15 days before expiration, and disable the user 7 days after expiration.
```
chage -M 100 -W 15 -I 7 hellokitty
```

Switch user - su.

[root ~]# su hellokitty
[hellokitty root]$

Execute the command as administrator - sudo.
```
[hellokitty ~]$ ls /root
ls: cannot open directory /root: Permission denied
[hellokitty ~]$ sudo ls /root
[sudo] password for hellokitty:
```
Note: if you want the user to execute commands as an administrator, you must appear in the sudoers list. The sudoers file is in the / etc directory. If you want to edit the file directly, you can also use the following command.

Edit sudoers file - visudo.

The editor used here is vi, and the knowledge about vi is explained later. Part of the document is as follows:

## Allow root to run any commands anywhere 
root    ALL=(ALL)   ALL

## Allows members of the 'sys' group to run networking, software, 
## service management apps and more.
# %sys ALL = NETWORKING, SOFTWARE, SERVICES, STORAGE, DELEGATING, PROCESSES, LOCATE, DRIVERS
## Allows people in group wheel to run all commands
%wheel  ALL=(ALL)   ALL

## Same thing without a password
# %wheel    ALL=(ALL)   NOPASSWD: ALL

## Allows members of the users group to mount and unmount the
## cdrom as root
# %users  ALL=/sbin/mount /mnt/cdrom, /sbin/umount /mnt/cdrom

## Allows members of the users group to shutdown this system
# %users  localhost=/sbin/shutdown -h now

Displays user and user group information - id.

Send a message to other users - write / wall.

Sender:

[root ~]# write hellokitty
Dinner is on me.
Call me at 6pm.

Receiver:

[hellokitty ~]$ 
Message from root on pts/0 at 17:41 ...
Dinner is on me.
Call me at 6pm.
EOF

View / set whether to receive messages sent by other users - mesg.

[hellokitty ~]$ mesg
is y
[hellokitty ~]$ mesg n
[hellokitty ~]$ mesg
is n

file system

File and path

Naming rules: the maximum length of the file name is related to the file system type. Generally, the file name should not exceed 255 characters. Although most characters can be used for the file name, it is best to use symbols such as English uppercase and lowercase letters, numbers, underscores and dots. Although spaces can be used in the file name, they should be avoided as far as possible. Otherwise, when entering the file name, you need to put the file name in double quotation marks or escape the spaces through \.
Extension: in Linux system, the file extension is optional, but the use of extension is helpful to understand the file content. Some applications need to identify files by extension, but more applications do not rely on the file extension, just as the file command does not determine the file type according to the extension when identifying files.
Hidden files: files starting with a dot are hidden files (invisible files) in Linux systems.

directory structure

/bin - binary of the basic command.
/Boot - static file for the boot loader.
/dev - device file.
/etc - configuration file.
/Home - the parent directory of the normal user's home directory.
/lib - shared library file.
/64 bit shared file library.
/lost+found - store unlinked files.
/media - automatically identifies the mount directory of the device.
/mnt - mount point where the file system is temporarily mounted.
/opt - optional plug-in package installation location.
/proc - kernel and process information.
/root - the home directory of the super administrator user.
/Run - stores what the system needs to run.
/sbin - superuser binary.
/sys - pseudo file system of the device.
/tmp - temporary folder.
/usr - user application directory.
/var - variable data directory.

Access rights

chmod - changes the file mode bit.
```
[root ~]# ls -l
...
-rw-r--r--  1 root       root 211878 Jun 19 16:06 sohu.html
...
[root ~]# chmod g+w,o+w sohu.html
[root ~]# ls -l
...
-rw-rw-rw-  1 root       root 211878 Jun 19 16:06 sohu.html
...
[root ~]# chmod 644 sohu.html
[root ~]# ls -l
...
-rw-r--r--  1 root       root 211878 Jun 19 16:06 sohu.html
...
```
Note: as can be seen from the above example, there are two ways to change the file mode bit with chmod: one is the character setting method, and the other is the number setting method. In addition to chmod, you can use umask to set which permissions will be deleted from the default permissions of the new file.

When viewing directories or files in long format, the display results and the corresponding permissions are shown in the following table.

chown - change the file owner.

[root ~]# ls -l
...
-rw-r--r--  1 root root     54 Jun 20 10:06 readme.txt
...
[root ~]# chown hellokitty readme.txt
[root ~]# ls -l
...
-rw-r--r--  1 hellokitty root     54 Jun 20 10:06 readme.txt
...

chgrp - change user groups.

Disk management

Lists the disk usage of the file system - df.

[root ~]# df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/vda1        40G  5.0G   33G  14% /
devtmpfs        486M     0  486M   0% /dev
tmpfs           497M     0  497M   0% /dev/shm
tmpfs           497M  356K  496M   1% /run
tmpfs           497M     0  497M   0% /sys/fs/cgroup
tmpfs           100M     0  100M   0% /run/user/0

Disk partition table operation - fdisk.

[root ~]# fdisk -l
Disk /dev/vda: 42.9 GB, 42949672960 bytes, 83886080 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000a42f4
   Device Boot      Start         End      Blocks   Id  System
/dev/vda1   *        2048    83884031    41940992   83  Linux
Disk /dev/vdb: 21.5 GB, 21474836480 bytes, 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk partitioning tool - parted.
Format file system - mkfs.
```
[root ~]# mkfs -t ext4 -v /dev/sdb
```
- -t - specifies the type of file system.
- -c - check for disk corruption when creating a file system.
- -v - display details.
File system check - fsck.
Convert or copy files - dd.
Mount / unmount - mount / umount.
Create / activate / close swap partition - mkswap / swap on / swap off.

Note: there are certain risks in executing the above commands. If you don't know the usage of these commands, you'd better not use them at will. In the process of use, you'd better refer to the reference materials and confirm whether you want to do so before operation.

Editor - vim

Start vim. You can start VIM through vi or VIM commands. You can specify a file name to open a file when starting. If you do not specify a file name, you can also specify a file name when saving.
```
[root ~]# vim guess.py
```
Command mode, editing mode and last line mode: start vim to enter the command mode (also known as Normal mode). In the command mode, entering the English letter i will enter the editing mode (INSERT mode), and a prompt -- INSERT -- appears at the bottom of the screen; Press Esc in the edit mode to return to the command mode. At this time, if you enter English: it will enter the last line mode, and enter q in the last line mode! You can forcibly exit vim without saving the current work; In the command mode, entering v will enter the Visual mode (Visual mode). You can select an area with the cursor and then complete the corresponding operation.
Save and exit vim: enter in the command mode: enter the last line mode, and enter wq to save and exit; If you want to discard the edited content, enter q! Forced exit, which has just been mentioned; In the command mode, you can also directly enter ZZ to save and exit. If you only want to save the file without exiting, you can enter w in the last line mode; You can enter a space after W and specify the file name to save.
Cursor operation.
- In the command mode, you can control the cursor to move left, down, up and right through h, j, k and l. you can enter a number in front of the letter to represent the moving distance. For example, 10h represents moving 10 characters to the left.
- In the command mode, you can use Ctrl+y and Ctrl+e to scroll up and down a line of text, and Ctrl+f and Ctrl+b to turn pages forward and backward.
- In the command mode, you can move the cursor to the end of the file by entering the English letter G, to the beginning of the file by gg, or to the specified line by entering a number in front of G.
Text operation.
- Delete: in the command mode, you can use dd to delete the entire line; You can add a number before dd to specify the number of rows to delete; D $can be used to delete from the cursor to the end of the line, or d0 can be used to delete from the cursor to the beginning of the line; If you want to delete a word, you can use dw; If you want to delete the full text, you can enter:% d (where: used to enter last line mode from command mode).
- Copy and paste: in command mode, yy can be used to copy the whole line; You can specify the number of copied rows by adding a number before yy; You can paste the copied content to the cursor by p.
- Undo and restore: enter u in the command mode to undo the previous operation; You can resume the undone operation by Ctrl+r.
- Sort contents: enter%! sort.
Find and replace.
- The search operation needs to enter / enter the last line mode and provide regular expressions to match the corresponding contents, such as: / doc. * \, Enter n to search forward or n to search backward.
- The replacement operation requires input: enter the last line mode and specify the search range, regular expression, replaced content and matching options, such as:: 1, $s / doc*/ Hello / gice, where:
  - g - global: global match.
  - i - ignore case: ignore case matching.
  - c - confirm: confirmation is required during replacement.
  - e - error: ignore the error.
Parameter setting: vim can be set after entering the last line mode.
- Set the number of spaces in Tab: set ts=4
- Set display / non display line number: set nu / set nonu
- Set highlight on / off syntax: syntax on / syntax off
- Set the display ruler (the row and column where the cursor is located): set ruler
- Set enable / disable search result highlighting: set hls / set nohls
  
  Note: if you want the above settings to take effect automatically every time you start vim, you need to write these settings to the in the user's home directory vimrc file.
Advanced skills
- Compare multiple files.
```
[root ~]# vim -d foo.txt bar.txt
```
- Open multiple files.
```
[root ~]# vim foo.txt bar.txt hello.txt
```
  After starting vim, only one window displays foo Txt, you can enter ls in the last line mode to view the three open files, or you can enter B < num > in the last line mode to display another file. For example, you can use: b 2 to set bar Txt is displayed. You can use: b 3 to set hello Txt is displayed.
- Split and switch windows.
  
  You can enter sp or vs in the last line mode to split the window horizontally or vertically, so that we can open multiple editing windows at the same time. You can switch editing windows by pressing Ctrl+w twice. Executing exit operation in one window will only close the corresponding window, and other windows will continue to remain.
- Mapping shortcut keys: some common operations can be mapped into shortcut keys under vim to improve work efficiency.
  - Example 1: in command mode, enter F4 to delete 10000 lines of code from the first line.
    
    :map <F4> gg10000dd.
    
    Example 2: input in edit mode__ Main is directly completed as if__ name__ == '__ main__':.
    
    :inoremap __main if __name__ == '__main__':
  Note: i in i nore map in example 2 above indicates that the mapped key is used in editing mode, and nor indicates that recursion is not allowed, which is very important. Otherwise, if the key itself appears in the content corresponding to the key, recursion will be triggered (equivalent to entering an endless loop). If you want the mapped shortcut key to take effect every time you start vim, you need to write the mapping to the in the user's home directory vimrc file.
- Record macros.
  - In command mode, enter qa to start recording macros (where a is the name of the register, or other English letters or numbers 0-9).
  - Perform your operations (cursor operations, editing operations, etc.) and these operations will be recorded.
  - If the recording is complete, press q to end the recording.
  - Play the macro through @ a (a is the name of the register just used). If you want to execute the macro multiple times, you can add a number in front of it, such as 100@a Indicates that the macro will be played 100 times.
  - You can try the following example to experience the operation of recording macros. This example comes from Harttle Land website , the website provides a lot of tips about vim. If you are interested, you can learn about it.

Software installation and configuration

Using package management tools

yum - Yellowdog Updater Modified.
- yum search: search for packages, such as yum search nginx.
- yum list installed: lists installed packages, such as yum list installed | grep zlib.
- yum install: install packages, such as yum install nginx.
- Yum remove: delete the package, for example, yum remove nginx.
- Yum update: update packages. For example, yum update can update all packages, while yum update tar only updates tar.
- Yum check update: check which packages can be updated.
- yum info: displays information about the package, such as yum info nginx.
rpm - Redhat Package Manager.
- Install package: RPM - IVH < packagename > rpm.
- Remove the package: RPM - e < packagename >.
- Query the software package: rpm -qa. For example, you can use rpm -qa | grep mysql to check whether MySQL related software packages are installed.

Next, take Nginx as an example to demonstrate how to install software using yum.

[root ~]# yum -y install nginx
...
Installed:
  nginx.x86_64 1:1.12.2-2.el7
Dependency Installed:
  nginx-all-modules.noarch 1:1.12.2-2.el7
  nginx-mod-http-geoip.x86_64 1:1.12.2-2.el7
  nginx-mod-http-image-filter.x86_64 1:1.12.2-2.el7
  nginx-mod-http-perl.x86_64 1:1.12.2-2.el7
  nginx-mod-http-xslt-filter.x86_64 1:1.12.2-2.el7
  nginx-mod-mail.x86_64 1:1.12.2-2.el7
  nginx-mod-stream.x86_64 1:1.12.2-2.el7
Complete!
[root ~]# yum info nginx
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
Installed Packages
Name        : nginx
Arch        : x86_64
Epoch       : 1
Version     : 1.12.2
Release     : 2.el7
Size        : 1.5 M
Repo        : installed
From repo   : epel
Summary     : A high performance web server and reverse proxy server
URL         : http://nginx.org/
License     : BSD
Description : Nginx is a web server and a reverse proxy server for HTTP, SMTP, POP3 and
            : IMAP protocols, with a strong focus on high concurrency, performance and low
            : memory usage.
[root ~]# nginx -v
nginx version: nginx/1.12.2

Remove Nginx.

[root ~]# yum -y remove nginx

Next, take MySQL as an example to demonstrate how to install software using rpm. To install mysql, you need to arrive first MySQL official website Download the corresponding RPM file Of course, choose the version corresponding to the Linux system you use. MySQL is now a product of Oracle. After MySQL was acquired, the author of MySQL recreated MariaDB, a branch of MySQL, which can be installed through yum.

[root mysql]# ls
mysql-community-client-5.7.22-1.el7.x86_64.rpm
mysql-community-common-5.7.22-1.el7.x86_64.rpm
mysql-community-libs-5.7.22-1.el7.x86_64.rpm
mysql-community-server-5.7.22-1.el7.x86_64.rpm
[root mysql]# yum -y remove mariadb-libs
[root mysql]# yum -y install libaio
[root mysql]#rpm -ivh mysql-community-common-5.7.26-1.el7.x86_64.rpm
...
[root mysql]#rpm -ivh mysql-community-libs-5.7.26-1.el7.x86_64.rpm
...
[root mysql]#rpm -ivh mysql-community-client-5.7.26-1.el7.x86_64.rpm
...
[root mysql]#rpm -ivh mysql-community-server-5.7.26-1.el7.x86_64.rpm
...

Note: due to MySQL and MariaDB The underlying dependency libraries of are in conflict, so we first removed the dependency library named MariaDB LIBS with yum and installed the dependency library named libaio to support asynchronous I/O operations. You can read about the relationship between MySQL and MariaDB Wikipedia Introduction to MariaDB on.

Remove the installed MySQL.

[root ~]# rpm -qa | grep mysql | xargs rpm -e

Download and extract configuration environment variables

Next, take MongoDB installation as an example to demonstrate how such software should be installed.

[root ~]# wget https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-rhel70-3.6.5.tgz
--2018-06-21 18:32:53--  https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-rhel70-3.6.5.tgz
Resolving fastdl.mongodb.org (fastdl.mongodb.org)... 52.85.83.16, 52.85.83.228, 52.85.83.186, ...
Connecting to fastdl.mongodb.org (fastdl.mongodb.org)|52.85.83.16|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 100564462 (96M) [application/x-gzip]
Saving to: 'mongodb-linux-x86_64-rhel70-3.6.5.tgz'
100%[==================================================>] 100,564,462  630KB/s   in 2m 9s
2018-06-21 18:35:04 (760 KB/s) - 'mongodb-linux-x86_64-rhel70-3.6.5.tgz' saved [100564462/100564462]
[root ~]# gunzip mongodb-linux-x86_64-rhel70-3.6.5.tgz
[root ~]# tar -xvf mongodb-linux-x86_64-rhel70-3.6.5.tar
mongodb-linux-x86_64-rhel70-3.6.5/README
mongodb-linux-x86_64-rhel70-3.6.5/THIRD-PARTY-NOTICES
mongodb-linux-x86_64-rhel70-3.6.5/MPL-2
mongodb-linux-x86_64-rhel70-3.6.5/GNU-AGPL-3.0
mongodb-linux-x86_64-rhel70-3.6.5/bin/mongodump
mongodb-linux-x86_64-rhel70-3.6.5/bin/mongorestore
mongodb-linux-x86_64-rhel70-3.6.5/bin/mongoexport
mongodb-linux-x86_64-rhel70-3.6.5/bin/mongoimport
mongodb-linux-x86_64-rhel70-3.6.5/bin/mongostat
mongodb-linux-x86_64-rhel70-3.6.5/bin/mongotop
mongodb-linux-x86_64-rhel70-3.6.5/bin/bsondump
mongodb-linux-x86_64-rhel70-3.6.5/bin/mongofiles
mongodb-linux-x86_64-rhel70-3.6.5/bin/mongoreplay
mongodb-linux-x86_64-rhel70-3.6.5/bin/mongoperf
mongodb-linux-x86_64-rhel70-3.6.5/bin/mongod
mongodb-linux-x86_64-rhel70-3.6.5/bin/mongos
mongodb-linux-x86_64-rhel70-3.6.5/bin/mongo
mongodb-linux-x86_64-rhel70-3.6.5/bin/install_compass
[root ~]# vim .bash_profile
...
PATH=$PATH:$HOME/bin:$HOME/mongodb-linux-x86_64-rhel70-3.6.5/bin
export PATH
...
[root ~]# source .bash_profile
[root ~]# mongod --version
db version v3.6.5
git version: a20ecd3e3a174162052ff99913bc2ca9a839d618
OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013
allocator: tcmalloc
modules: none
build environment:
    distmod: rhel70
    distarch: x86_64
    target_arch: x86_64
[root ~]# mongo --version
MongoDB shell version v3.6.5
git version: a20ecd3e3a174162052ff99913bc2ca9a839d618
OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013
allocator: tcmalloc
modules: none
build environment:
    distmod: rhel70
    distarch: x86_64
    target_arch: x86_64

Note: of course, you can also install MongoDB through yum. For details, please refer to Official website The instructions given on.

Source code build installation

Install Python 3.6.

[root ~]# yum install gcc
[root ~]# wget https://www.python.org/ftp/python/3.6.5/Python-3.6.5.tgz
[root ~]# gunzip Python-3.6.5.tgz
[root ~]# tar -xvf Python-3.6.5.tar
[root ~]# cd Python-3.6.5
[root ~]# ./configure --prefix=/usr/local/python36 --enable-optimizations
[root ~]# yum -y install zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel gdbm-devel db4-devel libpcap-devel xz-devel
[root ~]# make && make install
...
[root ~]# ln -s /usr/local/python36/bin/python3.6 /usr/bin/python3
[root ~]# python3 --version
Python 3.6.5
[root ~]# python3 -m pip install -U pip
[root ~]# pip3 --version

Note: after installing Python, you also need to register the PATH environment variable, and register the absolute PATH of the bin folder under the Python installation PATH into the PATH environment variable. Registered environment variables can modify the user's home directory bash_ The difference between profile or profile file in / etc directory is that the former is equivalent to user environment variable and the latter is equivalent to system environment variable.

Install Redis-3.2.12.

[root ~]# wget http://download.redis.io/releases/redis-3.2.12.tar.gz
[root ~]# gunzip redis-3.2.12.tar.gz
[root ~]# tar -xvf redis-3.2.12.tar
[root ~]# cd redis-3.2.12
[root ~]# make && make install
[root ~]# redis-server --version
Redis server v=3.2.12 sha=00000000:0 malloc=jemalloc-4.0.3 bits=64 build=5bc5cd3c03d6ceb6
[root ~]# redis-cli --version
redis-cli 3.2.12

Configure services

We can install and configure various services under the Linux system, that is, we can build the Linux system into database server, Web server, cache server, file server, message queue server, etc. Most services under Linux are set as daemons (running in the system background, but Linux cannot stop running because the service is still running). Therefore, the name of the service we installed usually has a letter d after it, which is the abbreviation of the English word daemon. For example, the firewall service is firewalld, and the MySQL service we installed before is mysqld, The Apache server is called httpd and so on. After installing the service, you can use systemctl command or service command to start and stop the service. The specific operations are as follows.

Start the firewall service.
```
[root ~]# systemctl start firewalld
```
Terminate the firewall service.
```
[root ~]# systemctl stop firewalld
```
Restart the firewall service.
```
[root ~]# systemctl restart firewalld
```
View firewall service status.
```
[root ~]# systemctl status firewalld
```

Set / disable firewall service startup.

[root ~]# systemctl enable firewalld
Created symlink from /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service to /usr/lib/systemd/system/firewalld.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/firewalld.service to /usr/lib/systemd/system/firewalld.service.
[root ~]# systemctl disable firewalld
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.

Planning tasks

Execute the command at the specified time.
- At - queue the task and execute it at the specified time.
- atq - view the queue of tasks to be executed.
- atrm - deletes the pending task from the queue.
Specify the tasks to be performed at 5 p.m. after 3 days.
```
[root ~]# at 5pm+3days
at> rm -f /root/*.html
at> <EOT>
job 9 at Wed Jun  5 17:00:00 2019
```
View the task queue to be executed.
```
[root ~]# atq
9       Wed Jun  5 17:00:00 2019 a root
```
Deletes the specified task from the queue.
```
[root ~]$ atrm 9
```

Schedule task table - crontab.

[root ~]# crontab -e
* * * * * echo "hello, world!" >> /root/hello.txt
59 23 * * * rm -f /root/*.log

Note: entering the crontab -e command will open vim to edit Cron expression and specify the triggered task. Above, we customized two scheduled tasks, one is to send hello. Com to / root directory every minute Txt, add the output hello, world!; The other is to delete files with log suffix in / root directory at 23:59 every day. If you don't know how to write Cron expressions, you can refer to the tips in the / etc/crontab file (described below) or use the search engine to find the "Cron expression online generator" to generate Cron expressions.

The crontab related files are in the / etc directory. You can also customize the scheduled tasks by modifying the crontab file in the / etc directory.

[root ~]# cd /etc
[root etc]# ls -l | grep cron
-rw-------.  1 root root      541 Aug  3  2017 anacrontab
drwxr-xr-x.  2 root root     4096 Mar 27 11:56 cron.d
drwxr-xr-x.  2 root root     4096 Mar 27 11:51 cron.daily
-rw-------.  1 root root        0 Aug  3  2017 cron.deny
drwxr-xr-x.  2 root root     4096 Mar 27 11:50 cron.hourly
drwxr-xr-x.  2 root root     4096 Jun 10  2014 cron.monthly
-rw-r--r--   1 root root      493 Jun 23 15:09 crontab
drwxr-xr-x.  2 root root     4096 Jun 10  2014 cron.weekly
[root etc]# vim crontab
  1 SHELL=/bin/bash
  2 PATH=/sbin:/bin:/usr/sbin:/usr/bin
  3 MAILTO=root
  4
  5 # For details see man 4 crontabs
  6
  7 # Example of job definition:
  8 # .---------------- minute (0 - 59)
  9 # |  .------------- hour (0 - 23)
 10 # |  |  .---------- day of month (1 - 31)
 11 # |  |  |  .------- month (1 - 12) OR jan,feb,mar,apr ...
 12 # |  |  |  |  .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat
 13 # |  |  |  |  |
 14 # *  *  *  *  * user-name  command to be executed

Network access and management

Secure remote connection - ssh.

[root ~]$ ssh root@120.77.222.217
The authenticity of host '120.77.222.217 (120.77.222.217)' can't be established.
ECDSA key fingerprint is SHA256:BhUhykv+FvnIL03I9cLRpWpaCxI91m9n7zBWrcXRa8w.
ECDSA key fingerprint is MD5:cc:85:e9:f0:d7:07:1a:26:41:92:77:6b:7f:a0:92:65.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '120.77.222.217' (ECDSA) to the list of known hosts.
root@120.77.222.217's password:

Get resources through the network - wget.
- -b background download mode
- -O download to the specified directory
- -r recursive Download
Send and receive mail - mail.

Network configuration tool (old) - ifconfig.

[root ~]# ifconfig eth0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.18.61.250  netmask 255.255.240.0  broadcast 172.18.63.255
        ether 00:16:3e:02:b6:46  txqueuelen 1000  (Ethernet)
        RX packets 1067841  bytes 1296732947 (1.2 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 409912  bytes 43569163 (41.5 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions

Network configuration tool (New) - ip.

[root ~]# ip address
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:16:3e:02:b6:46 brd ff:ff:ff:ff:ff:ff
    inet 172.18.61.250/20 brd 172.18.63.255 scope global eth0
       valid_lft forever preferred_lft forever

Network accessibility check - ping.

[root ~]# ping www.baidu.com -c 3
PING www.a.shifen.com (220.181.111.188) 56(84) bytes of data.
64 bytes from 220.181.111.188 (220.181.111.188): icmp_seq=1 ttl=51 time=36.3 ms
64 bytes from 220.181.111.188 (220.181.111.188): icmp_seq=2 ttl=51 time=36.4 ms
64 bytes from 220.181.111.188 (220.181.111.188): icmp_seq=3 ttl=51 time=36.4 ms
--- www.a.shifen.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 36.392/36.406/36.427/0.156 ms

Display or manage routing table - route.
View network services and Ports - netstat / ss.
```
[root ~]# netstat -nap | grep nginx
```
Network monitoring and packet capturing - tcpdump.
Secure file copy - scp.

[root ~]# scp root@1.2.3.4:/root/guido.jpg hellokitty@4.3.2.1:/home/hellokitty/pic.jpg

File synchronization tool - rsync.

Note: rsync can realize the automatic synchronization of files, which is very important for the file server. The usage of this command will be described in detail later when we talk about project deployment.
Secure file transfer - sftp.
```
[root ~]# sftp root@1.2.3.4
root@1.2.3.4's password:
Connected to 1.2.3.4.
sftp>
```
- Help: displays help information.
- ls/lls: displays a list of remote / local directories.
- cd/lcd: switch remote / local paths.
- mkdir/lmkdir: create remote / local directory.
- pwd/lpwd: displays the current remote / local working directory.
- get: download the file.
- put: upload file.
- rm: Delete remote files.
- bye/exit/quit: exit sftp.

Process management

ps - view the process.

[root ~]# ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  0 Jun23 ?        00:00:05 /usr/lib/systemd/systemd --switched-root --system --deserialize 21
root         2     0  0 Jun23 ?        00:00:00 [kthreadd]
...
[root ~]# ps -ef | grep mysqld
root      4943  4581  0 22:45 pts/0    00:00:00 grep --color=auto mysqld
mysql    25257     1  0 Jun25 ?        00:00:39 /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid

Displays the process status tree - pstree.

[root ~]# pstree
systemd─┬─AliYunDun───18*[{AliYunDun}]
        ├─AliYunDunUpdate───3*[{AliYunDunUpdate}]
        ├─2*[agetty]
        ├─aliyun-service───2*[{aliyun-service}]
        ├─atd
        ├─auditd───{auditd}
        ├─dbus-daemon
        ├─dhclient
        ├─irqbalance
        ├─lvmetad
        ├─mysqld───28*[{mysqld}]
        ├─nginx───2*[nginx]
        ├─ntpd
        ├─polkitd───6*[{polkitd}]
        ├─rsyslogd───2*[{rsyslogd}]
        ├─sshd───sshd───bash───pstree
        ├─systemd-journal
        ├─systemd-logind
        ├─systemd-udevd
        └─tuned───4*[{tuned}]

Find processes that match the specified criteria - pgrep.
```
[root ~]$ pgrep mysqld
3584
```

Terminate process by process number - kill.

[root ~]$ kill -l
 1) SIGHUP       2) SIGINT       3) SIGQUIT      4) SIGILL       5) SIGTRAP
 6) SIGABRT      7) SIGBUS       8) SIGFPE       9) SIGKILL     10) SIGUSR1
11) SIGSEGV     12) SIGUSR2     13) SIGPIPE     14) SIGALRM     15) SIGTERM
16) SIGSTKFLT   17) SIGCHLD     18) SIGCONT     19) SIGSTOP     20) SIGTSTP
21) SIGTTIN     22) SIGTTOU     23) SIGURG      24) SIGXCPU     25) SIGXFSZ
26) SIGVTALRM   27) SIGPROF     28) SIGWINCH    29) SIGIO       30) SIGPWR
31) SIGSYS      34) SIGRTMIN    35) SIGRTMIN+1  36) SIGRTMIN+2  37) SIGRTMIN+3
38) SIGRTMIN+4  39) SIGRTMIN+5  40) SIGRTMIN+6  41) SIGRTMIN+7  42) SIGRTMIN+8
43) SIGRTMIN+9  44) SIGRTMIN+10 45) SIGRTMIN+11 46) SIGRTMIN+12 47) SIGRTMIN+13
48) SIGRTMIN+14 49) SIGRTMIN+15 50) SIGRTMAX-14 51) SIGRTMAX-13 52) SIGRTMAX-12
53) SIGRTMAX-11 54) SIGRTMAX-10 55) SIGRTMAX-9  56) SIGRTMAX-8  57) SIGRTMAX-7
58) SIGRTMAX-6  59) SIGRTMAX-5  60) SIGRTMAX-4  61) SIGRTMAX-3  62) SIGRTMAX-2
63) SIGRTMAX-1  64) SIGRTMAX
[root ~]# kill 1234
[root ~]# kill -9 1234

Example: use a command to forcibly terminate the running Redis process.

ps -ef | grep redis | grep -v grep | awk '{print $2}' | xargs kill

Terminate process by process name - Kill / pkill.

End the process named mysqld.
```
[root ~]# pkill mysqld
```
End all processes of Hello Kitty user.
```
[root ~]# pkill -u hellokitty
```
Note: this operation will disconnect the Hello Kitty user from the server.
Put the process in the background.
- Ctrl+Z - shortcut key to stop the process and put it in the background.
- &- put the process in the background.
```
[root ~]# mongod &
[root ~]# redis-server
...
^Z
[4]+  Stopped                 redis-server
```

Query background process - jobs.

[root ~]# jobs
[2]   Running                 mongod &
[3]-  Stopped                 cat
[4]+  Stopped                 redis-server

Let the process continue running in the background - bg.

[root ~]# bg %4
[4]+ redis-server &
[root ~]# jobs
[2]   Running                 mongod &
[3]+  Stopped                 cat
[4]-  Running                 redis-server &

Put the background process in the foreground - fg.
```
[root ~]# fg %4
redis-server
```
Note: the process placed in the foreground can be terminated by Ctrl+C.
Adjust program / process runtime priority - nice / renice.
The process continues to work after the user logs out - nohup.
```
[root ~]# nohup ping www.baidu.com > result.txt &
```

Track process system calls - strace.

[root ~]# pgrep mysqld
8803
[root ~]# strace -c -p 8803
strace: Process 8803 attached
^Cstrace: Process 8803 detached
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 99.18    0.005719        5719         1           restart_syscall
  0.49    0.000028          28         1           mprotect
  0.24    0.000014          14         1           clone
  0.05    0.000003           3         1           mmap
  0.03    0.000002           2         1           accept
------ ----------- ----------- --------- --------- ----------------
100.00    0.005766                     5           total

Note: the usage and parameters of this command are complex. It is recommended that you understand it according to your actual needs when you really use this command.

View the current run level - runlevel.
```
[root ~]# runlevel
N 3
```

Real time monitoring of process resource usage - top.

[root ~]# top
top - 23:04:23 up 3 days, 14:10,  1 user,  load average: 0.00, 0.01, 0.05
Tasks:  65 total,   1 running,  64 sleeping,   0 stopped,   0 zombie
%Cpu(s):  0.3 us,  0.3 sy,  0.0 ni, 99.3 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem :  1016168 total,   191060 free,   324700 used,   500408 buff/cache
KiB Swap:        0 total,        0 free,        0 used.   530944 avail Mem
...

-c - displays the entire path of the process.
-d - specifies the interval between two swipes in seconds.
-i - idle or zombie processes are not displayed.
-p - displays information about the specified process.

System diagnosis

System startup exception diagnosis - dmesg.

View system activity information - sar.

[root ~]# sar -u -r 5 10
Linux 3.10.0-957.10.1.el7.x86_64 (izwz97tbgo9lkabnat2lo8z)      06/02/2019      _x86_64_        (2 CPU)

06:48:30 PM     CPU     %user     %nice   %system   %iowait    %steal     %idle
06:48:35 PM     all      0.10      0.00      0.10      0.00      0.00     99.80

06:48:30 PM kbmemfree kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit  kbactive   kbinact   kbdirty
06:48:35 PM   1772012   2108392     54.33    102816   1634528    784940     20.23    793328   1164704         0

-A - displays the health status of all devices (CPU, memory, disk).
-u - displays the load of all CPU s.
-d - displays the usage of all disks.
-r - displays memory usage.
-n - displays the network operation status.

View memory usage - free.

[root ~]# free
              total        used        free      shared  buff/cache   available
Mem:        1016168      323924      190452         356      501792      531800
Swap:             0           0           0

Virtual memory statistics - vmstat.

[root ~]# vmstat
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 2  0      0 204020  79036 667532    0    0     5    18  101   58  1  0 99  0  0

CPU Information Statistics - mpstat.

[root ~]# mpstat
Linux 3.10.0-957.5.1.el7.x86_64 (iZ8vba0s66jjlfmo601w4xZ)       05/30/2019      _x86_64_        (1 CPU)

01:51:54 AM  CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest  %gnice   %idle
01:51:54 AM  all    0.71    0.00    0.17    0.04    0.00    0.00    0.00    0.00    0.00   99.07

View process memory usage - pmap.

[root ~]# ps
  PID TTY          TIME CMD
 4581 pts/0    00:00:00 bash
 5664 pts/0    00:00:00 ps
[root ~]# pmap 4581
4581:   -bash
0000000000400000    884K r-x-- bash
00000000006dc000      4K r---- bash
00000000006dd000     36K rw--- bash
00000000006e6000     24K rw---   [ anon ]
0000000001de0000    400K rw---   [ anon ]
00007f82fe805000     48K r-x-- libnss_files-2.17.so
00007f82fe811000   2044K ----- libnss_files-2.17.so
...

Report device CPU and I/O Statistics - iostat.

[root ~]# iostat
Linux 3.10.0-693.11.1.el7.x86_64 (iZwz97tbgo9lkabnat2lo8Z)      06/26/2018      _x86_64_       (1 CPU)
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.79    0.00    0.20    0.04    0.00   98.97
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
vda               0.85         6.78        21.32    2106565    6623024
vdb               0.00         0.01         0.00       2088          0

Displays all PCI devices - lspci.

[root ~]# lspci
00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
00:01.2 USB controller: Intel Corporation 82371SB PIIX3 USB [Natoma/Triton II] (rev 01)
00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
00:02.0 VGA compatible controller: Cirrus Logic GD 5446
00:03.0 Ethernet controller: Red Hat, Inc. Virtio network device
00:04.0 Communication controller: Red Hat, Inc. Virtio console
00:05.0 SCSI storage controller: Red Hat, Inc. Virtio block device
00:06.0 SCSI storage controller: Red Hat, Inc. Virtio block device
00:07.0 Unclassified device [00ff]: Red Hat, Inc. Virtio memory balloon

Displays the status of the interprocess communication facility - ipcs.

[root ~]# ipcs

------ Message Queues --------
key        msqid      owner      perms      used-bytes   messages    

------ Shared Memory Segments --------
key        shmid      owner      perms      bytes      nattch     status      

------ Semaphore Arrays --------
key        semid      owner      perms      nsems

Shell Programming

As we mentioned earlier, shell is an application that connects users and the operating system. It provides a human-computer interaction interface (Interface), through which users access the services of the operating system kernel. Shell script is a script program written for shell. We can use shell script for system management and file operation. In short, writing shell scripts should be a standard skill for people using Linux systems.

There are a lot of knowledge about Shell scripts on the Internet. I don't intend to give a comprehensive and systematic explanation of Shell scripts here. Let's have a perceptual understanding of Shell scripts through the following code.

Example 1: enter two integers m and N to calculate the sum of integers from m to n.

#!/usr/bin/bash
printf 'm = '
read m
printf 'n = '
read n
a=$m
sum=0
while [ $a -le $n ]
do
    sum=$[ sum + a ]
    a=$[ a + 1 ]
done
echo 'result: '$sum

Example 2: automatically create folders and a specified number of files.

#!/usr/bin/bash
printf 'Enter folder name: '
read dir
printf 'Enter file name: '
read file
printf 'Enter the number of files(<1000): '
read num
if [ $num -ge 1000 ]
then
    echo 'The number of files cannot exceed 1000'
else
    if [ -e $dir -a -d $dir ]
    then
        rm -rf $dir
    else
        if [ -e $dir -a -f $dir ]
        then
            rm -f $dir
        fi
    fi
    mkdir -p $dir
    index=1
    while [ $index -le $num ]
    do
        if [ $index -lt 10 ]
        then
            pre='00'
        elif [ $index -lt 100 ]
        then
            pre='0'
        else
            pre=''
        fi
        touch $dir'/'$file'_'$pre$index
        index=$[ index + 1 ]
    done
fi

Example 3: automatically install the specified version of Redis.

#!/usr/bin/bash
install_redis() {
    if ! which redis-server > /dev/null
    then
        cd /root
        wget $1$2'.tar.gz' >> install.log
        gunzip /root/$2'.tar.gz'
        tar -xf /root/$2'.tar'
        cd /root/$2
        make >> install.log
        make install >> install.log
        echo 'installation is complete'
    else
        echo 'Already installed Redis'
    fi
}

install_redis 'http://download.redis.io/releases/' $1

Related resources

Common shortcut keys for Linux command line

Shortcut key	Function description
tab	Auto completion command or path
Ctrl+a	Move the cursor to the beginning of the command line
Ctrl+e	Move the cursor to the end of the command line
Ctrl+f	Move the cursor one character to the right
Ctrl+b	Move the cursor one character to the left
Ctrl+k	Cut the character from the cursor to the end of the line
Ctrl+u	Cut the character from the cursor to the beginning of the line
Ctrl+w	Cut a word in front of the cursor
Ctrl+y	Copy cut named cut content
Ctrl+c	Interrupt a task in progress
Ctrl+h	Deletes a character in front of the cursor
Ctrl+d	Exit the current command line
Ctrl+r	Search history command
Ctrl+g	Exit history command search
Ctrl+l	Clear everything on the screen and open a new line at the top of the screen
Ctrl+s	Lock the terminal so that it cannot input content temporarily
Ctrl+q	Exit terminal lock
Ctrl+z	Stop the task being executed by the terminal and put it in the background
!!	Execute the previous command
! number	Execute the historical command corresponding to the number
! letter	Execute the most recent command that begins with a letter
!$ / Esc+.	Get the last parameter of the previous command
Esc+b	Move to the beginning of the current word
Esc+f	Move to the end of the current word

Refer to the content description of the command manual

Title in manual	Function description
NAME	Description and introduction of the command
SYNOPSIS	Use the basic syntax of this command
DESCRIPTION	Use the detailed description of the command and the functions of various parameters. Sometimes these information will appear in OPTIONS
OPTIONS	Description of command related parameter options
EXAMPLES	Reference examples of using this command
EXIT STATUS	Exit status code at the end of the command, usually 0 indicates successful execution
SEE ALSO	Other commands or information related to commands
BUGS	Description of command related defects
AUTHOR	Introduction to the author of the command

To: git hub python-100-days. If you are offended, please contact to delete it

Programmer Think