Using shell+crontab to clean up logs regularly

Posted by Erestar on Tue, 28 Jan 2020 18:02:16 +0100

Background introduction

Scan the specified log directory every hour, empty the file directly after it exceeds 100M, and use it as a temporary means of operation and maintenance.

The source code is mainly composed of two functions:

  1. reviewLogs() determines the log size and clears it;
  2. getdir() determines whether it is a directory or a file. If it is a directory, perform iterative search;

usage method:

  1. Create review logs.sh under ~ / folder_sj directory (rename by yourself, and replace the same name in the shell source code) and paste it into the source code
  2. Line 3 from the bottom of the shell source code. Configure the path to be scanned. All files in the directory will be scanned and processed. Please note that it is not only *. log
  3. In the reviewLogs() function, 100 in if [$filesizem - GT 100] is the file size, and the unit is M. The value can be modified as needed
  4. After configuration, check whether the log is output regularly. The path of the log is
    LOG=~/folder_sj/review_logs_${TIMESTAMP}.log.

Shell source code

#!/bin/bash

TIMESTAMP=`date +%Y%m`
LOG=~/folder_sj/review_logs_${TIMESTAMP}.log
echo "Start to review all log files  at `date`." >>${LOG}


function reviewLogs(){
    filename=$1
    filesize=`ls -l $filename | awk '{ print $5 }'`
    filesizeM=`echo "sclae=2; $filesize/1024/1024" | bc`
    if [ $filesizeM -gt 100 ]
    then
      echo "$filename is too bigger!!"  >>${LOG}
      echo '' > $filename
      filesize1=`ls -l $filename | awk '{ print $5 }'`
      filesizeM1=`echo "sclae=2; $filesize1/1024/1024" | bc`
      echo "now the file size is ${filesizeM1} !!"  >>${LOG}
      echo "==========================="  >>${LOG}
    fi
}

function getdir(){
    for element in `ls $1`
    do
        dir_or_file=$1"/"$element
        if [ -d $dir_or_file ]
        then
            getdir $dir_or_file
        else
            reviewLogs $dir_or_file
        fi
    done
}

getdir '/opt/apache-tomcat-7.0.70_app/logs'

echo "Stop to review all log files  at `date`." >>${LOG}
echo "============================================================================" >>${LOG}

Crontab configuration source code

# /etc/crontab: system-wide crontab
# Unlike any other crontab you don't have to run the `crontab'
# command to install the new version when you edit this file
# and files in /etc/cron.d. These files also have username fields,
# that none of the other crontabs do.

SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

# m h dom mon dow user  command
00 */1 * * * root /root/folder_sj/review_logs.sh

Topics: crontab shell Apache Tomcat