File Finding and Text Processing

Posted by urb on Sun, 25 Aug 2019 15:14:11 +0200

  • Define a command alias that works for all users
    For example: lftps='lftp 172.168.0.1/pub'
    [root@localhost /]# echo "alias lftps='lftp 172.168.0.1/pub'" >> /etc/bashrc
    [root@localhost /]# source /etc/bashrc 
    [root@localhost /]# alias
    alias cp='cp -i'
    alias egrep='egrep --color=auto'
    alias fgrep='fgrep --color=auto'
    alias grep='grep --color=auto'
    alias l.='ls -d .* --color=auto'
    alias lftps='lftp 172.168.0.1/pub'
    alias ll='ls -l --color=auto'
    alias ls='ls --color=auto'
    alias mv='mv -i'
    alias rm='rm -i'
    alias which='alias | /usr/bin/which --tty-only --read-alias --show-dot --show-tilde'
  • Display rows in / etc/passwd file that do not end with / bin/bash
    [root@localhost /]# grep -v "/bin/bash$" /etc/passwd

  • Find rows with two or three digits in the / etc/passwd file
    [root@localhost /]# grep "\<[1-9]\{2,3\}\>" /etc/passwd

  • Display lines in / proc/meminfo file beginning with capitals or lowercase S; implemented in three ways
    (1)[root@localhost /]# grep  -i '^s' /proc/meminfo
    (2)[root@localhost /]# grep   '^[sS]' /proc/meminfo 
    (3)[root@localhost /]# grep -E "^(s|S)" /proc/meminfo 
    SwapCached:            0 kB
    SwapTotal:       2097148 kB
    SwapFree:        2097148 kB
    Shmem:              7980 kB
    Slab:              64752 kB
    SReclaimable:      28244 kB
    SUnreclaim:        36508 kB
  • Using echo to output an absolute path and egrep to get the path name,
    Similar to the result of executing dirname/etcpasswd
    [root@localhost /]# echo "/etc/sysconfig/netscript " |  egrep -o  '^/.*/'
    /etc/sysconfig/
  • Find the IP address in ifconfig and ask the result to show only the IP address
    [root@localhost /]# ifconfig  |  egrep -o  '([0-9]{1,3}\.){3}[0-9]{1,3}' 
    172.20.10.7
    255.255.255.240
    172.20.10.15
    127.0.0.1
    255.0.0.0
  • vim custom automatic indentation of four characters
    Open the file / etc/vim/vimrc and add it to save
    set tabstop=4
    set shiftwidth=4
  • Write scripts to automatically add three users and calculate the sum of the UIDs of the three users

    #!/bin/bash
    id user1 &> /dev/null || useradd user1
    id user2 &> /dev/null || useradd user2
    id user3 &> /dev/null || useradd user3
    
    user1_id=$(id -u user1)
    user2_id=$(id -u user2)
    user3_id=$(id -u user3)
    
    id_num=$[$user1_id + $user2_id + $user3_id]
    
    echo "The id sum is $id_num"
  • find usage:
    Usage:
    find [OPTIONS] [Find Start Path] [Find Conditions] [Processing Action]
    Find the start path: specify the specific search target start path; default is the current directory;
    Search conditions: the specified search criteria can be based on the criteria of file name, size, type, affiliation, permission and so on; the default is to find all files under the specified path;
    Processing actions: the operation of files that meet the search criteria, such as deletion, etc. The default is output to standard output.
    Find conditions:
    (1) Search for:

           -name  "pattern"
           -iname "pattern"
          glob-style wildcards; *,?, [], [^]
           - regex pattern: Find files based on regular expression patterns, matching is the entire path, not its name;

    (2) Search according to the affiliation of documents:

          - UseUSERNAME: Find all files belonging to the designated user of the master;
          - group GRPNAME: Find all files belonging to a specified group of groups;
          - uid UID: Find all files belonging to the UID specified by the master;
         - gid GID: Find all files of the GID specified by the subgroup;
          - nouser: Find files that do not belong to the owner;
          - nogroup: Find files that do not belong to a group;

    (3) Search according to the type of file:

     -type TYPE: 
        f: Ordinary Documents
        d: Catalog files
        l: Symbolic Link File
        b: Block device file
        c: Character device files
        p: Pipeline files
        s: socket file

    (4) Search according to the size of the file:

    -size [+|-]#UNIT
        //Common Units: k, M, G
         #UNIT: (#-1, #]
        -#UNIT: [0,#-1]
        +#UNIT: (#, oo)

    (5) Search by time stamp:

    In terms of "days":
        -atime  [+|-]#
            #: [#, #-1)
            -#: (#, 0]
            +#: (oo, #-1]
        -mtime
        -ctime
    
    In minutes:
        -amin
        -mmin
        -cmin

    (6) Search according to permission:

      -perm  [/|-]mode
          mode: exact permission matching;
          / mode: any one of the permissions of any kind of user (u,g,o) (r,w,x) meets the requirements.
              There is an "or" relationship between 9-bit permissions.
          - mode: every bit (r,w,x) of the permissions of each type of user (u,g,o) meets the requirements at the same time.
            There is a "and" relationship between the 9-bit permissions.

    Processing actions:

        - print: output to standard output; default action;
        - ls: Similar to execute the "ls-l" command on the files found, output the details of the files;
        - delete: delete the files found;
        - fls/PATH/TO/SOMEFILE: Save the long format information of all the files found to the specified file;
        - ok COMMAND {}: Execute commands represented by COMMAND for each file found; each operation is confirmed by the user;
       - exec COMMAND {}: Execute commands represented by COMMAND for each file found;
    Note: When find passes the found file path to the following command, it first finds all the qualified file paths and passes them to the following command at one time.
    But some commands can't accept long parameters, and then command execution will fail; another way can avoid this problem:
        find | xargs COMMAND
  • Examples of common uses of find:
    (1) Find all files or directories whose subordinates are root and whose subordinates are mail.
    [root@localhost /]# find /var -user root -a -group mail  -ls
    5002    0 drwxrwxr-x   2 root     mail          224 Aug 13 14:13 /var/spool/mail
    3360  152 -rw-------   1 root     mail       154132 Jul 19 16:12 /var/spool/mail/root

    (2) Find all files or directories that do not belong to root, bin or mysql in the / usr directory;

    [root@localhost /]# find /usr -not -user  root -a -not -user bin -a -not -user mysql
    /usr/share/polkit-1/rules.d
    /usr/libexec/abrt-action-install-debuginfo-to-abrt-cache
    (3)Find files or directories that do not belong to or belong to a group on the current system and have been accessed in the last week.
    [root@localhost /]# find  /  \( -nouser -o -nogroup \)  -atime  -7  -ls
    find: '/proc/2710/task/2710/fd/6': No such file or directory
    find: '/proc/2710/task/2710/fdinfo/6': No such file or directory
    find: '/proc/2710/fd/6': No such file or directory
    find: '/proc/2710/fdinfo/6': No such file or directory

    (4) Search for all files larger than 1M in the / etc directory with the type of ordinary files;

    [root@localhost /]# find /etc/ -size +1M -type f  -exec ls -lh {} \;
      -r--r--r--. 1 root root 7.2M Jul  8 17:32 /etc/udev/hwdb.bin
      -rw-r--r--. 1 root root 3.6M Aug  6  2017 /etc/selinux/targeted/active/policy.kern
      -rw-r--r--. 1 root root 1.4M Aug  6  2017 /etc/selinux/targeted/contexts/files/file_contexts.bin
      -rw-r--r--. 1 root root 3.6M Aug  6  2017 /etc/selinux/targeted/policy/policy.30
      -rw-r--r--. 1 root root 1.4M Aug  7  2017 /etc/brltty/zh-tw.ctb

    (5) Find files in the / etc directory where all users have no write permission;
    [root@localhost /]# find /etc -not -perm /222 -type f -ls
    (6) Find at least one class of files in the / etc directory that users do not have permission to execute;
    [root@localhost /]# find /etc/ -not -perm -111 -type f -ls
    (7) Find all files in / etc/init.d / directory where all users have execution rights and other users have write rights.
    [root@localhost /]# find /etc -perm -113 -type f -ls

Topics: Linux SELinux vim MySQL socket