Linux system | Linux system emergency response

Posted by exec1 on Thu, 13 Jan 2022 04:00:24 +0100

catalogue

Check user related information

Check the process port related information

Find malicious programs and kill them

cut the weeds and dig up the roots

Determine the intrusion mode and fix the vulnerability

When we were told that a Linux server was invaded by a hacker, the hacker used the server to mine, and placed a Trojan horse back door on the server. Now we need to check the server, close and remove the mining program and Trojan back door, find out how hackers invaded the server, and finally repair the vulnerability to ensure the normal operation of the server.

First of all, the first thing we do after logging in to the host should first use history to view the history commands of the host. Although most hackers delete the used commands after intrusion, it does not rule out that some hackers do not delete the used commands. If it is not deleted, we can know what the hacker has done through historical commands, so the follow-up work is very simple. Portal - >

https://blog.csdn.net/qq_36119192/article/details/95881479

Check user related information

whoami       #View current user
who          #View all users currently logged in to the system
w            #Display the list of users who have logged in to the system and display the instructions being executed by the user
users        #Displays the user list of all users currently logged in to the system
last         #View the users and information who have successfully logged in recently, and view the / var/log/wtmp file
lastb        #View the users and information who failed to log in recently, and view the / var/log/btmp file
lastlog      #The latest login information of all users in the system is displayed, and the / var/log/lastlog file is read
cat  /etc/passwd    #View user information
cat  /etc/passwd | grep /bin/bash       #View users who can log in to the system
awk -F: '$3==0{print $1}' /etc/passwd   #Check the super user (uid=0). If some hackers change the uid of an ordinary user to 0, the ordinary user also has super permission
awk '/\$1|\$6/{print $1}' /etc/shadow   #View users who can log on remotely
more /etc/sudoers | grep -v "^#\|^$" | grep "ALL=(ALL)"  #View users with sudo privileges

Related articles:

https://blog.csdn.net/qq_36119192/article/details/94396392

https://blog.csdn.net/qq_36119192/article/details/82228791

Check the process port related information

top                          #Dynamic view process
ls -l /proc/18176/exe        #View the executable of the process with PID 18176
lsof -p 18176                #View the files opened by the process with PID 18176
lsof -c sshd                 #View files opened by process sshd
lsof -i:33946                #View some processes corresponding to port 33946
ps -p PID -o lstart          #View the start time point of the process
netstat -pantu | grep 18176  #View the port connection and filter the lines containing 18176 to view the connected ports
fuser -n tcp 33946           #View the process PID corresponding to port 33946
ps aux  /  ps -ef            #Static view process
pstree                       #View process tree
ps aux --sort -pcpu          #View the process statically, ranking from high to low according to cpu usage
ps aux --sort -pmem          #View the process statically, ranking from high to low according to memory usage

Related articles:

https://blog.csdn.net/qq_36119192/article/details/82191051

Find malicious programs and kill them

From the above, we can see that the PID of the malicious process is 18176, the local port occupied by the process is 33946, and the process name is vvpKI1

When using the following command to find the executable program of the malicious process, it is found that the executable program of the malicious process has been deleted. It can be seen that the Trojan horse is a file free memory Trojan horse

ls -l /proc/18176/exe

View the files opened by the process. Sometimes, malicious programs will open some hidden dictionary libraries to blow up other hosts in the intranet

lsof -p 18176

Track abnormal process operation

strace -tt -T -e trace=all -p 1389       #Track the operation of 1389 process

Kill the malicious process, but in most cases, the process will restart because there is a daemon.

kill -9 18176

cut the weeds and dig up the roots

When we find that after killing the malicious process, the process will restart after a period of time, it indicates that the process has a daemon. At this time, the first thing we think of is the crontab scheduled task.

You can view scheduled tasks using the following command

crontab -l  perhaps  cat /var/spool/cron/root

We can also view the log files of scheduled tasks

more   /var/log/cron
#Look for malicious scripts in cron files 
/var/spool/cron/*    /etc/crontab          /etc/cron.d/*       /etc/cron.daily/* /etc/cron.hourly/*   /etc/cron.monthly/*   /etc/cron.weekly/   /etc/anacrontab     /var/spool/anacron/*

#View the files modified in the last 3 days in the specified directory
find /etc/ /usr/bin/ /usr/sbin/ /bin/ /usr/local/bin/  -type f -mtime -3 | xargs ls -la

#Sort by time to confirm whether any command has been replaced recently, which can be combined with rpm -Va command
ls -alt /usr/bin /usr/sbin /bin /sbin /usr/local/bin | rpm -Va>rpm.log

#Check whether there are abnormal startup items
cat /etc/rc.local | chkconfig --list

We can analyze the command of the scheduled task, find the source of the malicious file according to the command, and eliminate the root!

Another thing we should think of is / root / ssh/authorized_ Keys file. After most hackers invade, they will write the SSH private key in this file to facilitate direct login when logging in next time. Let's check the file. If there is a private key written by a hacker, delete it!

Check and kill viruses and rootkit s with tools

#Kill rootkit
chkrootkit (Download address-http://www.chkrootkit.org)rkhunter (download address)- http://rkhunter.sourceforge.net)

#Kill virus
clamav(Download address-http://www.clamav.net/download.html)

#Kill webshell
cloudwalker(Download address-http://github.com/chaitin/cloudwalker)

Determine the intrusion mode and fix the vulnerability

After the malicious programs and Trojans put by hackers are deleted, we can probably know how hackers invade the host, and then modify the vulnerability to prevent hackers from invading again.

View the / var/log/secure file

cat /var/log/secure* | grep Accepted   #View login success records
cat /var/log/secure* | grep Failed     #View log in failure records

grep "Accepted " /var/log/secure* | awk '{print $1,$2,$3,$9,$11}'  #View the date, user name and ip address of successful login
grep "Failed password for root" /var/log/secure | awk '{print $11}'  #Check which IPS are blasting the root account of the host
grep "Failed password" /var/log/secure | awk {'print $9'} | sort | uniq -c | sort -nr  #View user name dictionary

Generally speaking, there are the following situations when hackers invade the host:

  • Intrusion through unauthorized redis vulnerability (many mining programs use this)
  • ssh weak password brute force cracking
  • Web application vulnerability intrusion

Reference article: Remember a Linux Trojan horse cleaning process

Related article: Redis unauthorized access vulnerability

Removal and analysis of Linux mining virus

Performance monitoring, daemon and scheduled task management under Linux

Source: childe Xie's blog

Editor in charge: Zuo