1. What is Shell

Shell is a command line interpreter. It is a black window. When we issue some commands to Linux, Shell translates these commands into the actions that kernel can understand, then calls the kernel, operates the hardware, completes the operation, and then translates the signals into adult understandable forms.

Shell is also a powerful programming language.

In a narrow sense, shell is not a language. It is a bridge between the computer kernel and people. In a broad sense, shell is a programming language.

Each operating system has a different Shell.

2. Shell interpreter

The Shell parser provided by Linux includes

cat /etc/shells 

/bin/sh              yes bash A soft link (equivalent to windows (shortcut to)
                     Hard link: e.g B yes A Hard links, A Point to source file, access B Will directly access the source file, not through A
                     When the source file is deleted, the soft link cannot be used, and the hard link will not be affected

/bin/bash            linux Default interpreter

/sbin/nologin        The following can be understood as foreign languages, such as French, English, German, etc., which are not commonly used
/bin/dash
/bin/tcsh
/bin/csh

3. helloword of Shell script

A program written in an interpretive language is called a script. It is a collection of command lines, and the computer will translate it line by line.

Script to #/ The beginning of bin/bash means that every subsequent command is a bash command. Note that the comments of the Shell script begin with # but #/ bin/bash is not a one line comment. This sentence indicates that the bash interpreter needs to be used.

touch helloworld.sh
vim helloworld.sh

#!/bin/bash
echo "helloworld"

The first way to execute a script

bash helloworld.sh 

File permissions of linux

rw-rw-r-- 1-3 Permissions for the owner of the file, 4-6 Have group permissions for files, 7-9 Permissions for others

To give HelloWorld Execute permission of SH script

chmod +x helloworld.sh

The second way to execute the script: self execution of the script

./helloworld.sh 

The essence of the first execution method is that the bash parser helps execute the script, so the script itself does not need execution permission. The second execution method is that the script executes itself, so the execution permission is required.

4. Variables in Shell

All Shell variables are strings, that is, when defining variables, there is no need to declare types, and there can be no spaces on both sides of the equal sign.

a=10
echo $a

b=11
echo $a+$b

a="love you"

Undo variable

unset a

Declare static variables, which cannot be undone or modified. Generally, they are not used

readonly a=10

In the Shell, there are some system predefined variables, called environment variables, which can be used directly.

View environment variables

env

Special variable: $n

$n function description: n is a number, $0 represents the name of the script, $1 - $9 represents the first to ninth parameters. Parameters above ten need to be contained in braces, such as

touch parameter.sh 
vim parameter.sh

#!/bin/bash
echo $1
echo $2
echo $3

chmod +x parameter.sh

./parameter.sh aaaa bbbb cccc

Special variables:$#

$# Function Description: get the number of all input parameters, which is commonly used in loops.

vim parameter.sh
#!/bin/bash
echo $#

./parameter.sh a b c d e f g

Special variables: $*$@

$* function description: this variable represents all parameters in the command line, $* regards all parameters as a whole;

$@ Function Description: this variable also represents all parameters in the command line, but $@ treats each parameter differently

vim parameter.sh
echo $*
echo $@

./parameter.sh a b c d e f g

Special variable: $?

        $？ Function Description: displays the return status of the last executed command. If the value of this variable is 0, it proves that the last command was executed correctly; If the value of this variable is non-zero (which number is determined by the command itself), it proves that the execution of the previous command is incorrect, which is used for calls between multiple scripts.

5. Operator

$((expression)) "or" $[expression]

echo $S[(2+3)*4]

6. Conditional judgment

[condition] there should be a space before and after the condition or test condition

[ $a -eq $b ]
echo $?

Judgment condition

Comparison between two integers
= string comparison
-lt Less than( less than)			-le Less than or equal to( less equal)
-eq Equals( equal)				-gt Greater than( greater than)
-ge Greater than or equal to( greater equal)	-ne Not equal to( Not equal)

Judge according to file permissions
-r Have read permission( read)			-w Have write permission( write)
-x Have the authority to execute( execute)

Judge by document type
-f The file exists and is a regular file( file)
-e File exists( existence)		-d The file exists and is a directory( directory)

View all functions of test

man test

7. Process control

If judgment: if the first parameter is 1, output a, 2, output b, and other outputs 3. Where test can be replaced by []

if test $1 = 1
then
    echo a
elif test $1 = 2
then
    echo b
else
    echo c
fi

If the first parameter is a case, the output of b is 3.

case $1 in 
"1")
    echo a
    ;;
"2")
    echo b
    ;;
*)
    echo c
    ;;
esac

for loop: from 1 to 100

#!/bin/bash
sum=0
for ((i=1;i<=100;i++))
do
    sum=$[$sum+$i]
done
echo $sum

Enhanced for

for i in a b c
do 
    echo $i
done

while loop: from 1 to 100

#!/bin/bash
sum=0
i=1
while [ $i -le 100 ]
do
    sum=$[$sum+$i]
    i=$[$i+1]
done
echo $sum

8. Read console input

Function of read (option) (parameter): read assigns a value to a variable through console input, providing a way for people other than parameters to interact with the program, which is called console input.

#!/bin/bash
read i
echo $i

Adding the - p parameter allows the console to print some prompt information

#!/bin/bash
read -p "input something" i
echo $i

- t parameter, input within ten seconds

#!/bin/bash
read -p "input something in 10s" -t 10 i
echo $i

9. Function

Basic grammar, the contents in [] can be omitted

[ function ] funname[()]
{
	Action;
	[return int;]
}
funname

Customize a function aaa with two parameters

function aaa
{
i=$[$1+$2]
echo $1
return $1
}

After that, AAA 4 and 5 can be called. 4 and 5 are parameters. The return value returns a status code. 0 means correct and others mean error.

System function

Basename path name: function description. The basename command will delete all prefixes, including the last ('/') character, and then display the string.

dirname file absolute path: function description. Remove the file name (non directory Part) from the given file name containing the absolute path, and then return the remaining path (directory Part))

10. Shell tool

Four stream processing tools

        cut

Cut's job is to "cut". Specifically, it is used to cut data in the file. The cut command cuts bytes, characters, and fields from each line of the file and outputs them.

cut [option parameter] filename

Option parameters	    function
-f	        Column number, which column to extract
-d	        Separator to split the column according to the specified separator
-c	        Specify specific characters

Cut out the IP address of the virtual machine

ifconfig | grep Bcast | cut -d : -f 2 | cut -d " " -f 1

Take out the 20th and 30th characters

ifconfig | grep Bcast | cut -c 20,30

Take out the 20th to 30th characters

ifconfig | grep Bcast | cut -c 20-30

        sed

Sed is a stream editor that processes one line at a time. During processing, the currently processed line is stored in the temporary buffer, called "mode space", and then the contents in the buffer are processed with sed command. After processing, the contents of the buffer are sent to the screen. The next line is then processed and repeated until the end of the file. The contents of the file do not change unless redirection is used to store the output.

sed [option parameter] 'command' filename

Option parameters	    function
-e	        Directly in instruction line mode sed Action editing for.
-i	        Edit files directly

command	    Function description
a 	        newly added, a Can be followed by a string and appear on the next line
d	        delete          '2d' Delete the second line
s	        Find and replace     '2s/Replaced character/Replaced content/'

Replace inet in the second line with aaaa

ifconig | sed '2s/inet/aaaa/'

Delete the second to fifth lines

ifconfig | sed '2,5d'

Changing the 0 of the second and fifth lines to X without g will only change the first 0 of the second and fifth lines. When there are multiple commands, you need to add - e

ifconfig | sed -e '2s/0/X/g' -e '5s/0/X/g';

Change inet afddr to ip addr

ifconfig | sed 's/inet addr/ip addr/'

Regular expression: matches a string with a very distorted character.

Replace the inner part of the expression up to the regular expression

ifconfig | sed 's/inet[6]* addr/ip address/'

Six rules of regular expressions

\   Escape

^   The beginning of a line
    ^R------Express with R First line

$   Match the end of a line
    R$Express with R Ending line

*   Indicates that the previous sub formula matches 0 or more times, which is a greedy match
    Zo*-----express Z The beginning is followed by several o，The following three strings can be matched
            Z
            Zo
            Zooo

.   Match an arbitrary character
    .*Match any string

[]  Indicates that a character within a certain range is matched
    [6,8]------Match 6 or 8
    [a-z]------Match one a-z Characters between
    [a-z]*-----Match any alphabetic string

Save the number 6 and call it later with (). You can save 9 subexpressions and use \ 1-9 to complete the call.

ifconfig | sed 's/inet\([6]*\) addr/ip\1 address/'

The range expression can also use regular expressions to enclose the range with two \.

Replace 0 with X for lines beginning with R and beginning with T.

ifconfig | sed '/^ *R/,/^ *T/s/0/X/g'

Each shell environment is a process, and the shell can start many, and each shell environment is independent

When executing a script, it is executed in the sub environment, not in the current environment.

Use source to complete the current environment and execute a script:

source a.sh

        awk

A powerful text analysis tool, read the file line by line, slice each line with a space as the default separator, and then analyze the cut part.

awk [option parameter] 'pattern1{action1} pattern2{action2}...' filename

Pattern: refers to the content found by AWK in the data, which is the matching pattern

action: a series of commands executed when a match is found

Option parameters	function
-F	    Specifies the input file separator
-v	    Assign a user-defined variable

Print out lines starting with r

cp /etc/passwd ./

cat passwd | awk '/^r/{print}'       

Use - F to specify the cutting rule, and take: as the cutting rule

cat passwd | awk -F : '/^r/{print $4}'

awk built in variable

variable	        explain
FILENAME	file name
NR	        Number of records read
NF	        Number of fields for browsing records (number of columns after cutting)

View awk user manual

man awk

        sort

The sort command is very useful in Linux. It sorts files and outputs the sorting results as standard.

Sort (option) (parameter)

option	        explain
-n	    Sort by value
-r	    Sort in reverse order
-t	    Sets the separator character used when sorting
-k	    Specify the columns to sort

cat passwd | awk -F : '/^a/{print}' | sort -t : -k 3 -n -r