From Chapter 2 of the second edition of shell script actual combat to improve user commands
Script 14 formatting long lines
If you're lucky, the Unix system you use already contains the fmt command, which is very useful if you work with text every day. fmt can help you format your email or fill the available page width of your document with text lines.
But some Unix systems do not have fmt. This is especially true for legacy systems, which, if any, are usually only a fairly streamlined implementation.
As shown in code listing 2-2, in the short shell script, the nroff command can be used to realize long line automatic line folding and short line filling. This command is an integral part of Unix from the beginning, and it is also a shell script wrapper.
Code fmt
#!/bin/bash
# fmt -- a text formatting tool that can be used as a package container for nroff
# Add two usage options: - w X, specify the row width- h hyphens allowed
while getopts "hw:" opt;do # : indicates that a value is received after the w parameter
case $opt in
h ) hyph=1 ;;
w ) width="$OPTARG" ;;
esac
done
shift $(($OPTIND -1))
nroff << EOF
.ll ${width:-72}
.na
.hy ${hyph:-0}
.pl 1
$(cat "$@")
EOF
exit
Operation results
$ fmt -h -w 50 014-ragged.txt So she sat on, with closed eyes, and half believed herself in Wonderland, though she knew she had but to open them again, and all would change to dull reality--the grass would be only rustling in the wind, and the pool rippling to the waving of the reeds--the rattling teacups would change to tin- kling sheep-bells, and the Queen's shrill cries to the voice of the shepherd boy--and the sneeze of the baby, the shriek of the Gryphon, and all the other queer noises, would change (she knew) to the confused clamour of the busy farm-yard--while the lowing of the cattle in the distance would take the place of the Mock Turtle's heavy sobs. $ fmt 014-ragged.txt So she sat on, with closed eyes, and half believed herself in Wonderland, though she knew she had but to open them again, and all would change to dull reality--the grass would be only rustling in the wind, and the pool rippling to the waving of the reeds--the rattling teacups would change to tinkling sheep-bells, and the Queen's shrill cries to the voice of the shepherd boy--and the sneeze of the baby, the shriek of the Gryphon, and all the other queer noises, would change (she knew) to the confused clamour of the busy farm-yard--while the lowing of the cattle in the distance would take the place of the Mock Turtle's heavy sobs.
Script 15 backup when deleting files
One of the most common problems for Unix users is that they can't easily recover files or directories that have been deleted by mistake. There are neither user-friendly applications like Undelete 360 and WinUndelete, nor utilities that can easily browse and recover deleted files with one click as in OS X. As long as you enter rm filename and press enter, the file is gone.
One solution is to automatically create deleted files and directories silently Deleted files archive. With some clever tricks in the script (as shown in listing 2-5), the user is almost unaware of the process.
Code newrm
#!/bin/bash
# newrm -- an alternative to the existing rm command
# The script realizes the basic file reply function by creating a new directory in the user's home directory
# It can recover either directories or individual files. If the user specifies the - f option, the deleted files are not archived
# Important warning: you need to schedule cron jobs or perform similar clear archive directories. otherwise
# The system will not actually delete any files, which will cause insufficient disk space
archivedir="/.deleted_files"
realrm="$(which rm)"
copy="$(which cp) -R"
if [ $# -eq 0 ];then # Output usage error information by rm command
exec $realrm # The current shell will be replaced by / bin/rm; exec executes the string as a command
fi
# Parse all options to find the - f option
flags=""
while getopts "dfiPRrvW" opt;do
case $opt in
f ) exec $realrm "$@" ;; # The exec call causes the script to exit directly
* ) flags="$flags -$opt" ;; # Leave the other options to the rm command
esac
done
shift $(( $OPTIND - 1 ))
# Main script start
# ===================
# Ensure that $archivedir exists.
if [ ! -d $archivedir ];then # 1
if [ ! -w $HOME ];then
echo "$0 failed: can't create $archivedir in $HOME" >&2
exit 1
fi
mkdir $archivedir
chmod 700 $archivedir # 2. Please reserve the read / write / execute permissions of the privacy settings
fi
for arg ;do
newname="$archivedir/$(date "+%S.%M.%H.%d.%m").$(basename "arg")" # 3. File name of splicing backup; basename truncates the file name from the file directory
if [ -f "$arg" -o -d "$arg" ];then
$copy "$arg" "$newname"
fi
done
exec $realrm $flags "$@" # 4 replace the current shell with realrm
Operation results
You can set an alias to install the script so that when you enter rm, you actually run the script instead of the command / bin/rm. bash or ksh creates aliases as follows
alias rm=xxx/xxx/newrm # Use alias to create an alias for the command $ ls ~/.deleted-files ls: /Users/taylor/.deleted-files/: No such file or directory $ newrm file-to-keep-forever $ ls ~/.deleted-files/ 51.36.16.25.03.file-to-keep-forever
Script 16 processes the archiving of deleted files
A hidden directory containing deleted files has now appeared in the user's home directory. It would be very helpful if there was a script that allows users to choose between different versions of deleted files. However, it is not easy to solve all the problems involved, including not finding the specified file at all, finding multiple qualified deleted files, etc. If there are multiple matching files, should the script automatically pick the most recent file to recover? Or tell the user that there is more than one match? Or list all the different file versions for users to choose? The shell script unrm in listing 2-7 details how to do this
Code unrm
#!/bin/bash
# unrm -- finds the specified file or directory in the deleted file archive. If there are multiple
# If the matching results are sorted by timestamp, the results are listed, and the user specifies the one to recover.
archivedir="$HOME/.deleted-files"
realrm="$(which rm)"
move="$(which mv)"
dest=$(pwd)
if [ ! -d $archivedir ];then # -d determine whether it is a directory
echo "$0: No deleted files directory: nothing to unrm" >&2
exit 1
fi
cd $archivedir
# If no parameters are provided, only the list of deleted files is displayed
if [ $# -eq 0 ];then # 1 if the user does not specify a parameter, the conditional statement block is executed
echo "Contents of your deleted files archive (sorted by date):"
# 51.36. 16.25. 03. 51.36 in file to keep forever 16.25. 03. Delete
ls -FC | sed -e 's/\([[:digit:]][[:digit:]]\.\)\{5\}//G '- E's / ^ / /' # 2 - f list directory only - C output by column and sort vertically; Time and date information including point number in ls output
exit 0
fi
# Otherwise, it must be processed according to the mode specified by the user
# Let's see if there are multiple matches
# You can specify the name of the file or directory you want to restore as a parameter.
matches="$(ls -d *"$1" 2> /dev/null |wc -l)" # 3 -d display the directory as a file instead of the files under it; In order to ensure that ls can list files with spaces in the file name, the less common nested double quotation marks ($1 around) are used in the code, and the wildcard * can match the timestamp before the file name
if [ $matches -eq 0 ];then
echo "No match for \"$1\" in the deleted file archive." >&2
exit 1
fi
if [ $matches -gt 1 ];then # 4 if the specified file or directory name has more than one match
echo "More than one file or directory match in the archive:"
index=1
for name in $(ls -td *"$1");do # -t sort by time
datetime="$(echo $name |cut -c1-14|awk -F. '{print $5"/"$4 at "$3":"$2":"$1" }')" # 5 convert the timestamp part of the file name into the date and time when the file is deleted
filename="$(echo $name |cut -c16-)"
if [ -d $name ];then
filecount="$(ls $name |wc -l |sed 's/[^[:digit:]]//G ') "# 6 the script does not choose to display the size of the matching directory. It displays the number of files in the matching directory, which is more practical
echo " $index) $filename (contents = ${filecount} times, deleted = $datetime)"
else
size=" $(ls -sdk1 $name | awk '{print $1}')" # 7 - k represents the size of the file in k bytes; - 1 displays the file in columns
echo " $index) $filename (size = ${size}Kb, deleted = $datetime)"
fi
index=$((index + 1))
done
echo ""
read -p "Which version of $1 do you want to restore ('0' to quit)? [1] : " desired
if [ ! -z "$(echo $desired | sed 's/[[:digit:]]//g' ];then
echo "$0: Restore canceled by user: invalid input." >&2
exit 1
fi
if [ ${desired:=-1} -ge $index ];then
echo "$0: Restore canceled by user； index value too big." >&2
exit 1
fi
if [ $desired -lt 1 ];then
echo "$0: restore canceled by user." >&2
exit 1
fi
restore="$(ls -td1 *"$1" |sed -n "${desired}p")" # 8 as long as the customer selects a matching file or directory, find out the corresponding file name; - n option specifies the line number
if [ -e "$dest/$1" ];then # 9. Ensure that unrm will not overwrite the existing copy of the file, and the recovery of the file or directory is realized by calling mv
echo "\"$1\" already exists in this directory.Cannot overwrite." >&2
exit 1
fi
echo -n "Restoring file \"$1\" ... "
$move "$remove" "$dest/$1"
echo "done."
read -p "Delete the additional copies of this file? [y]" answer # 10 after the recovery is completed, the user can choose whether to delete other copies of the file
if [ ${answer:=y} ="y" ];then
$realrm -rf *"$1"
echo "deleted."
else
echo "additional copies retained."
fi
else
if [ -e "$dest/$1" ];then
echo "\"$1\" already exists in this directory. Cannot overwrite." >&2
exit 1
fi
restore="$(ls -d *"$1")"
echo -n "Restoring file \"$1\"..."
$move "$restore" "$dest/$1"
echo "done"
fi
exit 0
Operation results
# The nonparametric shell script unrm lists the files currently recoverable
$ unrm
Contents of your deleted files archive (sorted by date):
detritus this is a test
detritus garbage
# The shell script unrm with parameters attempts to recover the file
$ unrm
detritus More than one file or directory match in the archive:
1) detritus (size = 7688Kb, deleted = 11/29 at 10:00:12)
2) detritus (size = 4Kb, deleted = 11/29 at 09:59:51)
Which version of detritus should I restore ('0' to quit)? [1] : 0
unrm: Restore canceled by user.
refine on
When using this script, it is important to note that without any control or restriction, files and directories in deleted files will continue to grow. In order to avoid this situation, you can call the find command in the cron job to trim the deleted files, and use the -mtime option to filter out those files that have not been used for weeks. Said that 14 days of archiving is almost enough to meet most users, while avoiding wasting too much disk space.
Although the function has been implemented, there is still room for improvement in the user friendliness of the script. Consider adding some options, such as - l (restore to the latest version of the file) and - D (delete a copy of the file). Which option are you going to add and how to deal with it?
Script 17 record file deletion
You may just want to track the deletion on the recording system and do not intend to archive the deleted files. In code listing 2-10, the file deletion operation completed with the rm command is recorded in a separate file without reminding the user. This can be achieved by writing a script that acts as a wrapper. The wrapper is between the actual Unix command and the user, and provides the user with practical functions that the original command does not have.
Wrapper is a very powerful concept, which will appear again and again in this book.
Code logrm
#!/bin/bash
# logrm -- logs all file deletions (unless the - s option is specified)
removelog="tmp/remove.log"
if [ $# -eq 0 ];then # 1 if no parameter is provided, the usage description of the script is generated
echo "Usage: $0 [-s] list of files or directories" >&2
exit 1
fi
if [ "$1" = "-s" ];then # 2 check whether the parameter is - s
# Request silent operation Do not record operations.
shift # Move left as input parameter
else
echo "$(date):${USER}:$@" >>$removelog # 3 write the timestamp, user name and command to the file $removelog
fi
/bin/rm "$@" # 4. The parameters specified by the user are passed to rm
exit 0
# Modify rm alias
alias rm=logrm
Operation results
$ touch unused.file ciao.c /tmp/junkit $ logrm unused.file /tmp/junkit $ logrm ciao.c $ cat /tmp/remove.log Thu Apr 6 11:32:05 MDT 2017: susan: /tmp/central.log Fri Apr 7 14:25:11 MDT 2017: taylor: unused.file /tmp/junkit Fri Apr 7 14:25:14 MDT 2017: taylor: ciao.c
refine on
There is a potential log file permissions issue. File remove Log can either be written by everyone, in which case any user can use something like cat / dev / null > / TMP / remove Log to clear the log file; Or everyone can't write, so the script can't record any events. You can use setuid permission to let the script run with the same permissions as the log file. But this method has two problems. First of all, it's really a bad idea! Never run a script with setuid! Using setuid to run commands as a specific user, no matter who the specific user is, it may introduce security risks to the system. Secondly, you may encounter a situation where the user has permission to delete his own file, but the script does not have permission to delete it, because the valid uid set with setuid will be inherited by the rm command, which is about to happen. When users can't even delete their own files, it will cause great confusion.
If you use the ext2, ext3, or ext4 file system (which is common in Linux), there are other ways: use the chatr command to set the append only file permission for the log file, and then open the write permission to everyone, so there will be no danger. Another solution is to write log messages to syslog through the logger command. Using logger to delete records is very simple and straightforward, as shown below:
logger -t logrm "${USER:-LOGNAME}: $*"
This will add an entry to the syslog data stream that ordinary users cannot handle. Its contents include logrm, user name and specified command.
If you choose to use logger, you should check syslogd(8) to ensure that the priority is user Log record of notice. This is basically in the file / etc / syslogd Specified in conf
Script 18 displays the contents of the directory
The ls command seems meaningless in one place: when listing directories, ls either lists the files in them, or displays the number of disk blocks occupied by the subdirectories themselves (in 1024 bytes). The typical output of ls -l is as follows:
drwxrwxr-x 2 taylor taylor 4096 Oct 28 19:07 bin
But it's not much use! What we really want to know is how many files are in the directory. This is exactly what the script in listing 2-12 does. It generates a beautiful multi column list of files and directories, showing the size of the files and the number of files contained in the directory
Code formatdir
#!/bin/bash
# formatdir -- output the directory list in a friendly and practical format
# Be careful to make sure scriptbc(script#9) In the current path,
# Because it will be called multiple times in the script
# This function formats the file size in KB into KB, MB and BG to improve the continuity of output
. ./scriptbc
readablesize(){ # 1 # This function takes a value in kilobytes and converts it to the most formatted unit output
if [ $1 -ge 1048576 ];then
echo "$(scriptbc -p 2 $1 / 1048576)GB"
else [ $1 -ge 1024 ];then
echo "$(scriptbc -p 2 $1 / 1024)MB"
else
echo "${1}KB"
fi
}
# Main script
if [ $# -gt 1 ];then
echo "Usage: $0 [dirname]" >&2
exit 1
elif [ $# -eq 1 ];then # Specify a different directory? 2 allows the user to specify the directory, and then use the cd command to switch the working directory of the current shell to the specified location
cd "$@" # Switch to the specified directory.
if [ $? -ne 0 ];then # $? Returns the running status of the previous command
exit 1
fi
fi
for line in *;do
if [ -d "$file" ];then
size=$(ls "$file" |wc -l|sed 's/[^:digit:]]//g') # 3 calculate the number of files in the directory (excluding hidden files)
if [ $size -eq 1 ];then
echo "$file ($size entry)|"
else
echo "$file ($size entries)|"
fi
else
size="$(ls -sk "$file" |awk '{print $1}')"
echo "$file ($(readablesize $size))|" # 4 call readablesize using the sub shell generated by $()
fi
# First use sed to replace the space with three caret characters (^ ^ ^)
# After merging the paired lines using the xargs command, restore the spaces in them
# Finally, output the aligned two columns through awk
done| sed 's/ /^^^/g'|xargs -n 2|sed 's/\^\^\^/ / /g'|awk -F\|'{printf "%-39s %-39\n",$1,$2}' # %-39s only one character with a width of 39 - indicates that the left alignment is sufficient for space filling
exit 0
Operation results
$ formatdir ~ Applications (0 entries) Classes (4KB) DEMO (5 entries) Desktop (8 entries) Documents (38 entries) Incomplete (9 entries) IntermediateHTML (3 entries) Library (38 entries) Movies (1 entry) Music (1 entry) ...
refine on
There is a question worth considering: what if you meet a user who likes to use three consecutive hyphens in the file name? This naming method is quite rare. We checked a Linux system containing 116696 files, and there was not even a caret in all file names. But if it does, the script output will be messy. If you have concerns, you can choose to convert spaces into other character sequences that appear less in file names. Like four caret characters? Or five?
Script 19 locates the file by file name
The locate command is very useful in Linux system, but it can not be found in other Unix genres. It matches the regular expression specified by the user by searching the pre-established file name database. Have you ever thought about finding the Lord quickly Where is the cshrc file (master. Cshrc)? Let's see how locate does it:
$ locate .cshrc /.Trashes/501/Previous Systems/private/etc/csh.cshrc /OS9 Snapshot/Staging Archive/:home/taylor/.cshrc /private/etc/csh.cshrc /Users/taylor/.cshrc /Volumes/110GB/WEBSITES/staging.intuitive.com/home/mdella/.cshrc
You can see that in OS X system, the master The cshrc file is located in the directory / private/etc. The locate version written by ourselves will view all files on the disk when building the internal file index, whether the files are in the trash queue or in a separate disk volume, even hidden files. This approach has both advantages and disadvantages, which we will talk about soon.
Code mklocatedb
#!/bin/bash
# mklocatedb -- use the find command to build a database of locatedb. The user must run the script as root.
locatedb="/tmp/locate.db"
if [ "$(whoami)" != "root" ];then # Check whether the current user is root
echo "Must be root to run this command." >&2
exit 1
fi
find / -print > $locatedb # The print option uses \ n (newline character) to separate each file or directory name of the output
exit 0
Code locate
#!/bin/bash # Locate -- finds the specified style in the locate database locatedb="/tmp/locate.db" exec grep -i "$@" $localtedb
Operation results
$ sudo mklocatedb Password: ... Wait a little longer. ... # We can easily use ls to check the database file size, as shown below $ ls -l /tmp/locate.db -rw-r--r-- 1 root wheel 174088165 Mar 26 10:02 /tmp/locate.db # Now everything is ready to use locate to find the file $ locate -i solitaire /Users/taylor/Documents/AskDaveTaylor image folders/0-blog-pics/vista-searchsolitaire.png /Users/taylor/Documents/AskDaveTaylor image folders/8-blog-pics/windows-playsolitaire-1.png /usr/share/emacs/22.1/lisp/play/solitaire.el.gz /usr/share/emacs/22.1/lisp/play/solitaire.elc /Volumes/MobileBackups/Backups.backupdb/Dave's MBP/2014-04-03-163622/BigHD/ Users/taylor/Documents/AskDaveTaylor image folders/0-blog-pics/vista-searchsolitaire.png /Volumes/MobileBackups/Backups.backupdb/Dave's MBP/2014-04-03-163622/BigHD/ Users/taylor/Documents/AskDaveTaylor image folders/8-blog-pics/windows-playsolitaire-3.png # You can also use this script to confirm other interesting statistics about the system, such as how many C source code files are there: $ locate '\.c$' | wc -l 1479
refine on
It's easy to keep the database in a reasonable update state. Just like the built-in command locate in most systems, it's OK to arrange cron to run mklocatedb in the early morning of every night. You can also increase the update frequency according to the local usage mode. Like other scripts executed by root users, make sure that the script itself cannot be modified by non root users.
One improvement of this script is to let locate check its call status if no style or file is specified If DB does not exist, output a meaningful error message and abort the operation. According to the current writing method, the script outputs the standard grep error message, which is not very useful. More importantly, there is a major security issue in allowing users to access all file names in the system, including those they do not have permission to view, which we discussed earlier. The script #39 discusses how to improve the security of the script.