Preface

_This paper simply records the knowledge about Shell scripting.
Original video directions:
    Original video Click here

1. Shell Variables

(1) Definition:

• Variable name = variable value

  A=hello  # Define variable A whose value is hello
  echo $A  # Call the variable, note the money, "$"
  
  Be careful:
  	Call another form:
  		1. ${} 
  

• Cancel variable: unset A

(2) Rules for defining variables:

• Case sensitive, general uppercase
• Cannot have special symbols
• Single and double quotation marks for variable values with spaces
• Variable name cannot start with a number
• No spaces on either side of the equal sign, e.g. A = 3 is wrong

(3) Definition of variables:

1. Basic Ways

Variable name = variable value

A=1546
echo $A
 Output:
1546

2. Assigning command results to variables

`` (apostrophe) or $()

A=`hostname`
echo $A
 Output:
algo-System-Product-Name

B=`data +%F`
echo ${B}
Output:
2021-12-27

3. Interactively define variables:

read [options] variable name

option	function
-p	Define information to prompt the user
-n	Define the number of characters (limit input length)
-s	Do not display (user input)
-t	Define timeout

Usage 1: User-defined variable values
read name
harry
echo $name
harry
read -p "Input your name:" name
Input your name:tom
echo $name
tom

Usage 2: Variable values come from files
cat 1.txt 
10.1.1.1 255.255.255.0
read ip mask < 1.txt 
echo $ip
10.1.1.1
echo $mask
255.255.255.0

4. Define variables of type:

declare [option] variable name = variable value

option	function
-i	Consider variables as integers
-r	Define read-only variables
-a	Define normal arrays, view normal arrays
-A	Define an array of associations; View Associated Array
-x	Export variables through environment

(4) Classification of variables

1. Local Variables

	-  Current user-defined variables,==Current process is valid, other processes or subprocesses are invalid==

2. Environmental variables

-  The current process is valid, and child processes can also be used 
-  env: View current user environment variables

• set: View local, environment variables
• export: variable name = variable value, changing local variable to environment variable

3. Global Variables

-  All users are valid,/env/bashrc 

4. System variables

- shell Its name and function have been fixed in itself

Built-in variables	Meaning
$?	Status returned after the last command was executed; A status value of 0 indicates normal execution, while a non-zero indicates an execution exception or error
$0	Name of the program or script currently executing
$#	Number of parameters to follow the script
$*	All parameters after the script, with parameters as a whole output, separated by spaces between each variable parameter
$@	All parameters after the script, parameters are independent and all output
$1-$9	The position parameter after the script, $1 for the first position parameter, and so on
${10}-${n}	Extend the position parameter, the 10th position variable must be enclosed in {} braces (expanded by more than 2 digits)
$$	Process number of the current process, e.g. echo $$
$!	Last process number running in the background (current terminal)
!$	Invoke parameters in last command history

Two and Four Operations

• Arithmetic operations: By default, shell s can only support simple integer operations
• Operations: +, -, *, /,%(Remainder)

(1) Four operators

Expr remembers to add spaces such as expr 10-5, the multiplication sign needs to be escaped [the actual operation seems unnecessary, I can use it without escaping]

Expression	Give an example
$(())	echo $((1+1)) output 2
$[]	echo $[10-5] output 5
expr	Expr 10/5 Notice the space
let	n=1;let n+=1 is equivalent to let n=n+1

(2), i++ and ++ I

-  Has no effect on variables 

• Affects the value of the expression:
  • i++: assign before operation
  • ++ i: first compute, then assign

3. Conditional Judgment

(1) Grammatical Format

	- Format 1: test Conditional expression
	- Format 2: [ Conditional expression ]
	- Format 3:[[ Conditional expression ]] ，Supporting Regularity =~

Be careful:

- Format 2 and 3,[Spaces+Expression+Spaces]，There are spaces on both sides

(2) Parameters related to conditional judgment

1. Judging File Types

Judgement parameters	Meaning
-e	Determine whether a file exists (any type of file)
-f	Determine if a file exists and is a normal file
-d	Determine if a file exists and is a directory
-L	Determine if a file exists and is a soft-connected file
-b	Determine if the file exists and is a block device file
-S	Determine if the file exists and is a socket file
-c	Determine if the file exists and is a character device file
-p	Determine if the file exists and is a named pipe software
-s	Determine if a file exists and is not empty (has content)

2. Judging file permissions

Judgement parameters	Meaning
-r	Is it readable by the current user
-w	Is it writable by the current user
-x	Is it executable by the current user
-u	Is there suid, Advanced Rights Risk Bit
-g	Is sgid, Advanced Rights Enforcement Bit
-k	Is there a t bit, advanced permissions sticky bit

3. Judging the New and Old Documents [Modification Time of Files]

Judgement parameters	Meaning
file1 -nt file2	Compare file1 to file2
file1 -ot file2	Compare file1 to file2
file1 -ef file2	Compare whether the same file, or a hard link, points to the same inode

4. Common Integers

parameter	Meaning
-eq	Equal
-ne	unequal
-gt	greater than
-lt	less than
-ge	Greater than or equal to
-le	Less than or equal to

5. Common judgment strings

Judgement parameters	Meaning
-z	Determines whether an empty string is valid with a string length of 0
-n	Determines if the string is not empty and the length of the string is not 0
string1 = string2	Determine whether strings are equal
string1 != string2	Determine whether strings are not equal

6. Multiple Judgements

Logical Operators, Left to Right

Symbol	Meaning
-a and &&	Logical and
-o and | |	Logical or

Special instructions:

- &&   Previous Expressions==True==，To execute the following code 
- | |     Previous Expressions==False==，To execute the following code
- ；     ==only==Be used for==Division==Command or expression

IV. if statement

(1) if structure: fi end sign

if [ condition ];then
	command
	command
fi
 or
[ condition ] && command

(2) if...else structure

Bifurcation, alternative

if [ condition ];then
	command
else
	command
fi
 or
[ condition ] && command1 || command2

(3) if...elif...else structure

There are many choices, only one can go

if [ condition1 ];then
	command1
elif[ condition2 ];then
	command2
else
	command3
fi

(4) Layer-by-layer nested structure

if [ condition1 ];then
	command1
	if [ condition2 ];then
		command2
else
	if [ condition3 ];then  
		command3
	elif [ condition4 ];then
		command4
	else
		command5
fi

5. Circular statements

(1) for cycle

Use occasion: known number of cycles

1. Grammatical Structure

1), List Loop

for variable in {list}
	do 
		command
		command
		...
	done
 perhaps
for variable in a b c
		do 
		command
		...
	done

2) Loop without list

		- Number of loops user specified

for variable
	do 
		command
		command
		...
	done

3) Class C Style

for ((expr1;expr2;expr3))
	do 
		command
		command
		...
	done
expr1: Define variables and assign initial values
expr2: Decide whether to cycle (condition)
expr3: Determines how the loop variable changes and when the loop exits
for(( i=1;i<=5;i++))
	do
		echo $i
	done

(2) while statement

1. Grammatical Structure

# Common way
 Define Variables; while[ Expression ];do program;let i++;done
# Class C Style
 Define Variables;while((Expression));do program;let i++;done

(3) until statements

False conditions enter the loop, true exits the loop

1. Grammatical Structure

until expression [ 1 -ed -1] (( 1 >= 1))
	do 
		command
		...
	done

6. Regular expressions

(1) Class I regular

• Regular Expression, regex, or regeexp, abbreviated RE)

• Metacharacters: e.g., dots, Star*, question mark? Wait.

• Leading characters: Characters that precede metacharacters

1. Common

Symbol	explain	case
.	Match any single character except line breaks
*	Leading characters occur 0 or more times in a row
.*	Any length character	ab.*
^	Beginning of line (beginning with)	^root
$	End of line (end with)	bash$
^$	Blank Line
[]	Match any single character or group of single characters in parentheses	[abc]
[^]	Match does not contain any single or a group of single characters in parentheses	[^abc]
1	Match begins with any single character or group of single characters in parentheses	2
3	Matching does not begin with any single character or group of single characters in parentheses	4

2. Other commonly used metacharacters

Symbol	explain	case
<	Take the head of a word
>	Take the end of a word
< >	Exact match
{n}	n consecutive occurrences of matching leading characters
{n, }	Match leading characters at least n times
{n, m}	Match leading characters between n and m occurrences
( )	Save matched characters
\d	Match Number (grep-P)	[0-9]
\w	Match Character Numeric Underline (grep-P)	[a-zA-Z0-9_]
\s	Match spaces, tabs, page breaks (grep-P)	[\t\r\n]

3. Regular metacharacters for extended classes

• grep must be added-E to use, or egrep to use
• sed must be added-r

Symbol	explain	case
+	Match one or more leading characters
？	Match zero or one leading character
|	or
()	Group characters (as a whole)
{n}	Leading Character Repeated n Times
{n,}	Leading character repeats at least n times
{n,m}	Leading character repeats n to m times

(2) Class II regular

Expression	function
[:alnum:]	Letter and numeric characters
[:alpha:]	Alphabetic characters (including case)
[:blank:]	Spaces and Tabs
[:digit:]	number
[:lower:]	Lowercase letters
[:upper:]	Capital
[:punct:]	Punctuation
[:space:]	All blanks including line breaks, carriage returns, etc.

Seven. sed command

(1) Introduction

• sed reads the contents of a file line by line and processes them as required, outputting the results to the screen
• Source files will not be modified directly

(2) Method of use

1. Command Line Format

- sed [option] =='==Processing Action=='== file name

Common Options

option	Meaning
-e	Multiple (multiple) edits
-n	Cancel default output
-r	Use extended regular expressions
-i	Edit in place (modify file)
-f	Specify sed script file name

Common actions

Note: All actions are in single quotes.

action	Meaning
'p'	Print
'i'	Insert content before specified line
'a'	Insert content after specified line
'c'	Replace all contents of specified line
'd'	Delete the specified line

Search for and replace files

- sed [option] `'s/Search Content/replace content/action'` Files to be processed
- p Print; Global Replacement g

command	explain
r	Read from another file
w	Save Content As
&	Save the lookup string to reference in the replacement string
=	Print line numbers
!	Apply commands to all lines except the selected lines, after the number of lines
q	Sign out

2. Script mode

• sed -f sacripts.sh file
• First line of script: #!/ Bin/sed-f

Matters needing attention:

Only one script file sed Command line list.'commands'
You cannot have any spaces or tabs at the end of each line. tab)Or other text.
If you have more than one command on a line, apply semicolon division.
Unneeded and unavailable quotation mark protection commands
#Behavior Notes Beginning with Number

8. awk command

(1) Overview

1. Overview

• Programming language for processing text and data
• Scan files line by line to find rows that match a specific pattern
• gawk is the GUN version of awk

2. What can awk do?

• awk is used to process files and data.
• Can be used for statistics, such as site visits, IP visits
• Supports conditional judgment, for and while loops

(2) Use

1. Command Line Mode

• Grammatical structure:

  awk option 'Command section' file name
   Special instructions:
  	Quote shell Variables need to be used==Double quotation mark causes==
  

• Introduction to Common Options

  command	function
  -F	Definition character is divided by symbol, default separator is space
  -v	Define variables and assign values

• 'Named Section Description'

  • Regular expression, address location

    '/root/{awk Sentence}'   sed Medium:'/root/p'
    

  • {awk statement 1; awk statement 2;...}

    '{print $0; print $1}'   sed Medium:'p'
    

  • BEGIN...END...

    'BEGIN{awk Sentence};{Processing};END{awk Sentence}'
    'BEGIN{awk Sentence};{Processing}'
    '{Processing};END{awk Sentence}'
    

2. Script mode

• Scripting

  #!/bin/awk -f   # Define magic characters
   The following are awk List of commands in quotation marks, do not protect commands with quotation marks, multiple commands are separated by semicolons
  BEGIN{FS=":"}
  NR==1,NR==3{print $1"\t"$NF}
  

• Script Execution

  Method 1:
  awk option -f awk The text file to be processed by the script file of
  awk -f awk.sh filename
   Method 2:
  ./awk The text file to be processed by the script file (or absolute path) of
  ./awk.sh filename
  

(3) Internal related variables

1. Common built-in variables

variable	Variable Description	Remarks
$0	All records of the current processing line
$1,$2,$3...$n	Different fields in a file that are symbolically split between each line	awk -F: '{print $1,$3}' 1.txt
NF	Number of fields (columns) in the current record	awk -F: '{print NF}' 1.txt
$NF	Last Column	$(NF-1) indicates the second last line
FNR/NR	Line Number
FS	Define Interval Character	'BEGIN{FS=":"};{print $1,$3}'
OFS	Define output field delimiter, default space
RS	Enter record separator, default line break
ORS	Output record delimiter, default line break
FILENAME	File name currently entered

(4) Working principle

awk -F: '{print $1,$3}' /etc/passwd

1. awk takes a line as input and assigns it to an internal variable of $0, each of which can be called a record and ends with a line break (RS);

2. Each row is separated by an interval character: (default is space or tab) into fields (or fields), each stored in a numbered variable starting at $1;

   Q: How does awk know to split fields with spaces?

   A: Because there is an internal variable FS to determine the character segment separator. Initially, FS is given a space

3. awk uses the print function to print fields, which are separated by spaces because there is a comma between 1 and 3. It maps to another internal variable, called the output field delimiter OFS, which defaults to a space.

4. When awk finishes processing a line, it gets another line from the file, stores it in $0, overwrites the original, and then splits the new string into fields and processes it. The process continues until all rows are processed.

(5) Advance

1. Format output prints and printf s

print function     Similar echo
# data | awk '{print "Month: "$2"\nYear: "$NF}'
# awk -F: '{print "username is: " $1 "\t uid is: "$3}' /etc/passwd
printf function     Similar echo -n 
# awk -F: '{print "%-15s %-10s %-15s\n", $1, $2, $3}' /etc/passwd
# awk -F: '{print "|%15s| %10s| %15s|\n", $1, $2, $3}' /etc/passwd
# awk -F: '{print "|%-15s| %-10s| %-15s|\n", $1, $2, $3}' /etc/passwd


%s Character type
%d value type
- Indicates left alignment, defaults to right alignment
printf By default, no line wraps at the end, plus \n

2. awk variable definition

# Awk-v NUM=3-F:'{print $NUM}'/etc/passwd [Note]
# awk -v NUM=3 -F: '{print NUM}' /etc/passwd
# awk -v num=1 'BEGIN{print num}'
1
# Awk-v num=1'BEGIN{print $num}'[Note]

Be careful:
	awk The variable defined in the call does not need to be added $

3. BEGIN...END use in awk

• BEGIN: Represents execution before program starts
• END: Indicates that all files are processed and executed
• Usage:'BEGIN{before processing starts}; {In Process}; END{After Processing}'

4. awk's scripting

(1) if structure

format:
awk option 'Regular, Address Location{awk Sentence}' file name
{if(Expression){Statement 1; Statement 2;...}}

Case:
# awk -F: '{if($3>=500 && $3<=60000) {print $1,$3}}' passwd

# awk 'BEGIN{if($(id -u)==0){print "admin"}}'
admin

(2) if...else structure

Format:
{if(Expression){Sentence; Sentence;...}} else {Sentence; Sentence;...}}
awk -F: '{if($3>=500 && $3<=60000) {print $1"Is the average user"} else {print $1"Not an average user"}}' passwd

awk 'BEGIN{if($(id -u)>=500 && $(id -u) !=65534) {print "Is the average user"} else {print "Not an average user"}}'

(3) if...elif...else structure

Format:
{if(Expression 1) {Sentence; Sentence;...} elif(Expression 2) {Sentence; Sentence;...} elif(Representation 3) {Sentence; Sentence;...} else {Sentence; Sentence;...}}

awk -F: '{if($3==0) {print $1"Is the average user"} elif($3>=1 && $3<=499) {print $1"Not an average user"}}' passwd

10. export command

(1) What is the export command?

(2) Why should the export command be used?

• In order for us to be able to define a variable in a sub shell without having to redefine it.

(3) How to use the export command

1. Interpretation:

2. Grammar:

• export [options] variable name = variable value

3. Parameter description:

option	explain
-f	Represents the name of the function in [variable name]
-n	Delete the specified variable, note: the variable is not actually deleted, knowledge will not be output to the execution environment of subsequent instructions.
-p	List all environment variables that the shell assigns to the program

3. Conclusion

• When a script is executed, it runs in a sub shell environment, which exits automatically after the script is executed.
• System environment variables in a shell are copied into the child shells (variables defined by export);
• System environment variables in a shell are valid only for that shell or its child shells, and at the end of the shell the variables disappear (and cannot be returned to the parent shell).
• Variables that are not defined by export are valid only for that shell and also for child shells.

1. ↩︎

2. abc ↩︎

3. ^ ↩︎

4. ^abc ↩︎