Usually, when people mention "Shell Scripting Language", bash, ksh, sh or other similar linux/unix scripting language comes to mind. Scripting language is another way to communicate with computers. Using the graphical window interface (regardless of windows or linux), users can move the mouse and click on various objects, such as buttons, lists, checkboxes, etc. However, this method is very inconvenient every time the user wants the computer / server to complete the same task (such as batch conversion of photos, or downloading new movies, mp3, etc.). To make all these things simple and automated, we can use shell scripts.
Some programming languages, such as pascal, foxpro, C and java, need to be compiled before execution. They need the right compiler to get our code to do a task.
Other programming languages, such as php, javascript and visual basic, do not need a compiler, so they need an interpreter, and we can run the program without compiling code.
A shell script is also like an interpreter, but it is usually used to call an external compiled program. It then captures the output, exits the code, and processes it as appropriate.
Bash is one of the most popular shell scripting languages in the Linux world. I think (this is my own view) the reason is that by default, the bash shell allows users to easily navigate through historical commands (previously executed). On the contrary, ksh requires Make some adjustments to the profile, or remember some "magic" key combinations to review the history and correct the commands.
Well, I think these introductions are enough. You can judge which environment is most suitable for you. From now on, I'll just talk about Bash and its scripts. In the following example, I will use CentOS 6.6 and bash-4.1.2. Please make sure you have the same version or higher.
Shell script flow
shell scripting language is similar to chatting with several people. You just need to think of all the orders as those who can help you, as long as you ask them to do it in the right way. For example, you want to write documents. First, you need paper. Then, you need to tell someone the content and let him write it for you. Finally, you want to store it somewhere. Or you want to build a house, so you need to ask the right person to empty the site. When they say "it's done", other engineers can help you build the wall. Finally, when these engineers also tell you "it's done", you can ask the painter to paint the house. What would happen if you asked the painter to paint the wall before it was built? I think they'll start complaining. Almost all these human like commands can speak, and if they finish the work without any problems, they will tell "standard output". If they can't do what you ask them to do - they will tell "standard error". In this way, finally, all commands listen to you through "standard input".
Quick example - when you open a linux terminal and write some text - you are talking to bash through "standard input". So let's ask bash shell who am i (who am i?) All right.
root@localhost ~]# Who am I < --- you say to bash shell through standard input root pts/0 2015-04-22 20:17 (192.168.1.123) <--- bash shell Answer you through standard output
Now, let's say something bash doesn't understand:
[root@localhost ~]# BLA < --- ha, you're talking to standard input again -bash: blablabla: command not found <--- bash Complaining through standard errors
The first word before ":" is usually an order to complain to you. In fact, each of these streams has its own index number:
- Standard input (stdin) – 0
- Standard output (stdout) – 1
- Standard error (stderr) – 2
If you really want to know what the output command says - you need to redirect that speech to (use the greater than sign ">" and stream index after the command) file:
[root@localhost ~]# blablabla 1> output.txt -bash: blablabla: command not found
In this case, we try to redirect stream 1 (stdout) to a stream named output Txt file. Let's look at what we do with the contents of the file. We can do this with the cat command:
[root@localhost ~]# cat output.txt [root@localhost ~]#
It looks empty. Well, now let's redirect stream 2 (stderr):
[root@localhost ~]# blablabla 2> error.txt [root@localhost ~]#
Well, we see the complaint gone. Let's check that file:
[root@localhost ~]# cat error.txt -bash: blablabla: command not found [root@localhost ~]#
right enough! We see that all complaints are recorded in errors Txt file.
Sometimes the command produces both stdout and stderr. To redirect them to different files, we can use the following statement:
command 1>out.txt 2>err.txt
To shorten the statement, we can ignore "1", because stdout will be redirected by default:
command >out.txt 2>err.txt
Well, let's try to do something "bad". Let's delete file1 and folder1 with rm command:
[root@localhost ~]# rm -vf folder1 file1 > out.txt 2>err.txt
Now check the following output file:
[root@localhost ~]# cat out.txt removed `file1' [root@localhost ~]# cat err.txt rm: cannot remove `folder1': Is a directory [root@localhost ~]#
As we can see, different streams are separated into different files. Sometimes, this is not very convenient, because we want to see what happens continuously before or after some operations when an error occurs. To achieve this, we can redirect two streams to the same file:
command >>out_err.txt 2>>out_err.txt
Note: Please note that I use "> >" instead of ">". It allows us to attach to the file instead of overwriting the file.
We can also redirect one stream to another:
command >out_err.txt 2>&1
Let me explain. The standard output of all commands will be redirected to out_err.txt, the error output will be redirected to stream 1 (as explained above), and the stream will be redirected to the same file. Let's look at this example:
[root@localhost ~]# rm -fv folder2 file2 >out_err.txt 2>&1 [root@localhost ~]# cat out_err.txt rm: cannot remove `folder2': Is a directory removed `file2' [root@localhost ~]#
Looking at the output of these combinations, we can explain it as follows: first, the rm command tries to delete folder2, but it will not succeed, because linux requires the - r key to allow the rm command to delete the folder, and the second file2 will be deleted. By providing the - v (details) key for rm, we let the rm command tell us each deleted file or folder.
That's all you need to know about redirection. I mean, almost, because there is a more important redirection tool, which is called "pipeline". By using the | (pipe) symbol, we usually redirect stdout streams.
For example, we have a text file:
[root@localhost ~]# cat text_file.txt This line does not contain H e l l o word This lilne contains Hello This also containd Hello This one no due to HELLO all capital Hello bash world!
We need to find some lines with "Hello". There is a grep command in Linux to do this:
[root@localhost ~]# grep Hello text_file.txt This lilne contains Hello This also containd Hello Hello bash world! [root@localhost ~]#
This works well when we have a file and want to search it. What if we need to find something in the output of another command? Yes, of course, we can redirect the output to a file and then look in the file:
[root@localhost ~]# fdisk -l>fdisk.out [root@localhost ~]# grep "Disk /dev" fdisk.out Disk /dev/sda: 8589 MB, 8589934592 bytes Disk /dev/mapper/VolGroup-lv_root: 7205 MB, 7205814272 bytes Disk /dev/mapper/VolGroup-lv_swap: 855 MB, 855638016 bytes [root@localhost ~]#
What if you're going to grep some double quotes to cause content with spaces!
Note: the fdisk command displays information about the Linux operating system disk drives.
As we can see, this method is very inconvenient because we soon messed up the temporary file space. To accomplish this task, we can use pipes. They allow us to redirect the stdout of one command to the stdin stream of another:
[root@localhost ~]# fdisk -l | grep "Disk /dev" Disk /dev/sda: 8589 MB, 8589934592 bytes Disk /dev/mapper/VolGroup-lv_root: 7205 MB, 7205814272 bytes Disk /dev/mapper/VolGroup-lv_swap: 855 MB, 855638016 bytes [root@localhost ~]#
As you can see, we did not need any temporary files to get the same results. We redirected fdisk stdout to grep stdin.
Note: pipe redirection is always left to right.
There are several other redirects, but we'll talk about them later.
Display custom information in shell
As we know, communication with and within the shell is usually conducted in the form of dialogue. So let's create some real scripts that will also talk to us. This will let you learn some simple commands and have a better understanding of the concept of script.
Suppose we are the general service desk manager of a company. We want to create a shell script to register the call information: phone number, user name and brief description of the problem. We intend to store this information in a plain text file data Txt for future statistics. The script itself works in the way of dialogue, which will make the small life of the staff at the front desk easier. Well, first we need to show the question. For displaying information, we can use echo and printf commands. Both of them are used to display information, but printf is more powerful because we can format the output well through it. We can align it right, left or leave special space for information. Let's start with a simple example. To create a file, use your favorite text editor (kate, nano, vi,......), and then create a file named note SH file, in which these commands are written:
echo "Phone number ?"
How do I run / execute scripts?
After saving the file, we can run it with bash command and take our file as its parameter:
[root@localhost ~]# bash note.sh Phone number ?
In fact, it is inconvenient to execute the script in this way. It would be more comfortable not to use the bash command as a prefix. To make the script executable, we can use the chmod command:
[root@localhost ~]# ls -la note.sh -rw-r--r--. 1 root root 22 Apr 23 20:52 note.sh [root@localhost ~]# chmod +x note.sh [root@localhost ~]# ls -la note.sh -rwxr-xr-x. 1 root root 22 Apr 23 20:52 note.sh [root@localhost ~]#
Note: the ls command displays the files in the current folder. By adding the - la key, it displays more file information.
As we can see, before the chmod command is executed, the script has only read (r) and write (w) permissions. After executing chmod +x, it gets execute (x) permission. (more details about permissions will be covered in the next article.) Now we just need to run this:
[root@localhost ~]# ./note.sh Phone number ?
Before the script name, I added/ Combination (DOT) in the unix world means the current location (current folder), and / (slash) is the folder separator. (in Windows, we use backslash / to indicate the same function) therefore, the whole combination means "execute note.sh script from the current folder". I think if I run this script with the full path, you will know better:
[root@localhost ~]# /root/note.sh Phone number ? [root@localhost ~]#
It also works.
If all linux users have the same default shell, then everything is OK. If we just execute the script, the default user shell will be used to parse the script content and run commands. The syntax and internal commands of different shells are slightly different, so in order to ensure that our scripts will use bash, we should add #/ bin/bash to the first line of the file. In this way, the default user shell will call / bin/bash, and only then will the commands in the script be executed:
[root@localhost ~]# cat note.sh #!/bin/bash echo "Phone number ?"
Until now, we are only 100% sure that bash will be used to parse our script content. Let's continue.
Read input
After the message is displayed, the script waits for the user to answer. There is a read command to receive the user's answer:
#!/bin/bash echo "Phone number ?" read phone
After execution, the script will wait for user input until the user presses [ENTER] to end the input:
[root@localhost ~]# ./note.sh Phone number ? 12345 <--- Here is what I entered [root@localhost ~]#
Everything you enter will be stored in the variable phone. To display the value of the variable, we can also use the echo command:
[root@localhost ~]# cat note.sh #!/bin/bash echo "Phone number ?" read phone echo "You have entered $phone as a phone number" [root@localhost ~]# ./note.sh Phone number ? 123456 You have entered 123456 as a phone number [root@localhost ~]#
In bash shell, we usually use the $(dollar) symbol to indicate that this is a variable. We don't use this $(which will be explained in the future) except when we read in variables and a few others.
OK, now we are ready to add the remaining questions:
#!/bin/bash echo "Phone number?" read phone echo "Name?" read name echo "Issue?" read issue [root@localhost ~]# ./note.sh Phone number? 123 Name? Jim Issue? script is not working. [root@localhost ~]#
Using stream redirection
Perfect! The rest is to redirect everything to the file data Txt. As a field separator, we will use the / (slash) symbol.
Note: you can choose any separator you think is the best, but make sure that the file content does not include these symbols, otherwise it will cause additional fields in the text line.
Don't forget to use "> >" instead of ">", because we want to attach the output to the end of the file!
[root@localhost ~]# tail -2 note.sh read issue echo "$phone/$name/$issue">>data.txt [root@localhost ~]# ./note.sh Phone number? 987 Name? Jimmy Issue? Keybord issue. [root@localhost ~]# cat data.txt 987/Jimmy/Keybord issue. [root@localhost ~]#
Note: the tail command displays the last n lines of the file.
Done. Let's run it again to see:
[root@localhost ~]# ./note.sh Phone number? 556 Name? Janine Issue? Mouse was broken. [root@localhost ~]# cat data.txt 987/Jimmy/Keybord issue. 556/Janine/Mouse was broken. [root@localhost ~]#
Our files are growing. Let's add a date to each line, which will be very useful for manipulating these statistics in the future. To achieve this, we can use the date command and specify a format, because I don't like the default format:
[root@localhost ~]# date Thu Apr 23 21:33:14 EEST 2015 <---- date Default output of command [root@localhost ~]# date "+%Y.%m.%d %H:%M:%S" 2015.04.23 21:33:18 <---- Formatted output
There are several ways to read the output of the command to the variable. In this simple case, we will use ` (it is a backquote, not a single quote, and the tilde ~ is in the same key position):
[root@localhost ~]# cat note.sh #!/bin/bash now=`date "+%Y.%m.%d %H:%M:%S"` echo "Phone number?" read phone echo "Name?" read name echo "Issue?" read issue echo "$now/$phone/$name/$issue">>data.txt [root@localhost ~]# ./note.sh Phone number? 123 Name? Jim Issue? Script hanging. [root@localhost ~]# cat data.txt 2015.04.23 21:38:56/123/Jim/Script hanging. [root@localhost ~]#
Well... Our script looks a little ugly. Let's beautify it. If you want to read the read command manually, you will find that the read command can also display some information. To achieve this function, we should use the - p key with the information:
[root@localhost ~]# cat note.sh #!/bin/bash now=`date "+%Y.%m.%d %H:%M:%S"` read -p "Phone number: " phone read -p "Name: " name read -p "Issue: " issue echo "$now/$phone/$name/$issue">>data.txt
You can find a lot of interesting information about each command directly from the console. Just enter: man read, man echo, man date, man
Do you agree? It looks much more comfortable!
[root@localhost ~]# ./note.sh Phone number: 321 Name: Susane Issue: Mouse was stolen [root@localhost ~]# cat data.txt 2015.04.23 21:38:56/123/Jim/Script hanging. 2015.04.23 21:43:50/321/Susane/Mouse was stolen [root@localhost ~]#
It's interesting that the cursor is behind the message (not in a new line). (LCTT translation note: if the echo command is used to output the display, the - n parameter can be used to avoid line breaks.)
loop
It's time to improve our script. If users answer the phone all day, wouldn't it be troublesome if they have to run it every time? Let's cycle these activities endlessly:
[root@localhost ~]# cat note.sh
#!/bin/bash
while true
do
read -p "Phone number: " phone
now=`date "+%Y.%m.%d %H:%M:%S"`
read -p "Name: " name
read -p "Issue: " issue
echo "$now/$phone/$name/$issue">>data.txt
doneI've swapped the positions of the read phone and now=date lines. This is because I want to get time after entering the phone number. If I put it on the first line of the loop, the variable now will get the time immediately after the data is stored in the file. This is not good, because the next call may be 20 minutes later, or even later.
[root@localhost ~]# ./note.sh Phone number: 123 Name: Jim Issue: Script still not works. Phone number: 777 Name: Daniel Issue: I broke my monitor Phone number: ^C [root@localhost ~]# cat data.txt 2015.04.23 21:38:56/123/Jim/Script hanging. 2015.04.23 21:43:50/321/Susane/Mouse was stolen 2015.04.23 21:47:55/123/Jim/Script still not works. 2015.04.23 21:48:16/777/Daniel/I broke my monitor [root@localhost ~]#
Note: to exit from an infinite loop, you can press [Ctrl]+[C]. The Shell will display ^ for the CTRL key.
Using pipe redirection
Let's add more features to our Frankenstein. I want the script to display some statistics after each call. For example, I want to check each number and call me several times. For this, we should use the cat file data txt:
[root@localhost ~]# cat data.txt 2015.04.23 21:38:56/123/Jim/Script hanging. 2015.04.23 21:43:50/321/Susane/Mouse was stolen 2015.04.23 21:47:55/123/Jim/Script still not works. 2015.04.23 21:48:16/777/Daniel/I broke my monitor 2015.04.23 22:02:14/123/Jimmy/New script also not working!!! [root@localhost ~]#
Now, we can redirect all the output to the cut command, let cut cut each line into pieces (we use the separator "/"), and then print the second field:
[root@localhost ~]# cat data.txt | cut -d"/" -f2 123 321 123 777 123 [root@localhost ~]#
Now, we can redirect the output to another command sort:
[root@localhost ~]# cat data.txt | cut -d"/" -f2|sort 123 123 123 321 777 [root@localhost ~]#
Then leave only one row. To count unique entries, simply add the - c key to the uniq command:
[root@localhost ~]# cat data.txt | cut -d"/" -f2 | sort | uniq -c
3 123
1 321
1 777
[root@localhost ~]#Just add this to the end of our loop:
#!/bin/bash
while true
do
read -p "Phone number: " phone
now=`date "+%Y.%m.%d %H:%M:%S"`
read -p "Name: " name
read -p "Issue: " issue
echo "$now/$phone/$name/$issue">>data.txt
echo "===== We got calls from ====="
cat data.txt | cut -d"/" -f2 | sort | uniq -c
echo "--------------------------------"
donefunction:
[root@localhost ~]# ./note.sh
Phone number: 454
Name: Malini
Issue: Windows license expired.
===== We got calls from =====
3 123
1 321
1 454
1 777
--------------------------------
Phone number: ^C
The current scenario runs through several well-known steps:
- display messages
- Get user input
- Store values to file
- Processing stored data
However, if the user has a sense of responsibility, he sometimes needs to input data, sometimes needs statistics, or may need to find something in the stored data? For these things, we need to use switches/cases and know how to format the output well. This is useful when "drawing" tables in the shell