Hello, I'm 😎

I wrote an article on hydrology earlier< Obtain the ip address and mac address of all connected devices under the current LAN >But what I didn't expect was that I was on the hot list, which was also my first article on the hot list, and the reading volume soared instantly 💥. However, few people have seen my hard core technology article. Since many people are interested in this topic, we will continue to dig deeply into the relevant principles. It's best to realize it by ourselves and understand it thoroughly.

First, let's review the previous article. In the previous article, I introduced the command to obtain the ip address and arp mapping table under windows. By analyzing the latest arp mapping table, we can know the online or offline devices under the current network segment ⭐.

The technology used in this paper is to update the arp table by calling the system ping command in python. However, when the system's own ping command accesses the ip of the entire network segment, it takes up to 2 minutes. Later, it can only be accelerated to 25 seconds through multithreading. This speed delay is too large to be applied to more advanced applications 😇.

Today, our goal is to reduce the total time taken to Ping the IP of the whole network segment to less than 5 seconds, so that we can know the online and offline of the specified mac address device within 5 seconds. For example, we can develop a fishing artifact when the BOSS comes. As long as the BOSS's mobile phone is connected to wifi, we receive a notice within 5 seconds and stop fishing immediately, which ensures that we can fish safely and boldly at ordinary times ⚡.

So how to speed up? After several days of hard thinking and learning some network knowledge, I realized the PING command and successfully realized the safe and bold fishing. So, after reading several books, writing thousands of lines of code and stepping on hundreds of pits, I finally understood the relevant knowledge. The following is my summary of the core knowledge points involved into this article, so this article is very concise dry goods, which is very strong ❤️ Suggested collection ❤️.

After learning this article, your strength will not only stop here, but also be able to develop any custom protocol based on IP protocol. Of course, it depends on whether you have the ability to draw inferences from one instance. You can even continue to dig deep and develop protocols at a lower level than IP protocols.

Desire? Eager, then learn ⁉️ The following is the list of knowledge points of this article:

🎥 socket core knowledge

📚 socket introduction 🔥

Interprocess communication refers to the data sharing between running programs. On a computer, a process can be uniquely identified by process number (PID) for communication.

In the network, the "ip address" of the network layer of TCP/IP protocol family can uniquely identify the host in the network, while the "protocol + port" of the transport layer can uniquely identify the application process (process) in the host. The process communication in the network can interact with other processes through the flags of ip address, protocol and port.

Socket (socket for short) is a way to realize the communication between network processes. Most of the various services on the network are based on socket to complete the communication. In order to establish a communication channel, each endpoint of network communication has a socket socket object, which allows programs to accept and connect, such as sending and receiving data.

📹 socket link 🔥

Using the socket function of the socket module in Python can complete:

import socket
socket.socket(family=-1, type=-1, proto=-1, fileno=None)

Parameter Description:

Family is the specified address family. There are three main types:

• socket.AF_UNIX: used for interprocess communication on the same machine
• socket.AF_INET: Internet interprocess communication based on ipv4 protocol
• socket.AF_INET6: Internet interprocess communication based on ipv6 protocol

More address families include socket AF_ Bluetooth, socket AF_ Vsock virtual machine communication, socket AF_ Packet is directly connected to the underlying interface of network equipment, etc.

Type is the specified socket type. There are three main types:

• socket.SOCK_STREAM: stream socket, which uses connection oriented TCP protocol to realize byte stream transmission
• socket.SOCK_DGRAM: datagram socket, which uses non connection oriented UDP to realize datagram socket
• socket.SOCK_RAW: raw socket that allows direct access to lower layer protocols such as IP or ICMP

More socket types include socket SOCK_ RDM and socket SOCK_ Seqpacket al.

💾 TCP and UDP communication model 🔥

tcp or udp sockets can be created directly in the following ways:

import socket

# Create tcp socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Create udp socket
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
# Close the socket when not in use
s.close()

**UDP communication model: * * before communication, you don't need to establish relevant links, you just need to send data, which is similar to writing a letter:

UDP server example code:

from socket import *
# Create socket
udp_socket = socket(AF_INET, SOCK_DGRAM)
# Bind local related information. If not, the system will assign it randomly
udp_socket.bind(('0.0.0.0', 8080))
# Waiting to receive the data sent by the other party
recv_data = udp_socket.recvfrom(1024) #  1024 indicates the maximum number of bytes received this time
# Displays the received data. The first element is the data sent by the other party, and the second element is the ip and port of the other party
print(recv_data[0].decode('u8'))
# Close socket
udp_socket.close()

UDP client example code:

from socket import *
# Create udp socket
udp_socket = socket(AF_INET, SOCK_DGRAM)
# Send data to the specified program on the specified computer
udp_socket.sendto("Hello, server~".encode('u8'), ('192.168.1.103', 8080))
# Close socket
udp_socket.close()

**TCP communication model: * * before communication, you must establish relevant links before sending data, which is similar to calling:

TCP server example code:

from socket import *

# Create socket
tcp_server_socket = socket(AF_INET, SOCK_STREAM)
# Server binding native ip and port
tcp_server_socket.bind(('0.0.0.0', 8080))
# Listening port, 128 indicates that 128 client links can be received at the same time
tcp_server_socket.listen(128)
# If there is a new client to link to the server, a new socket is generated to serve the client
client_socket, clientAddr = tcp_server_socket.accept()
# Receive data sent by the other party
recv_data = client_socket.recv(1024)  # Receive 1024 bytes
print('The data received is:', recv_data.decode('u8'))
# Send some data to the client
client_socket.send("Hello, client!".encode('u8'))
# Close the socket serving this client
client_socket.close()

TCP client example code:

from socket import *

# Create socket
tcp_client_socket = socket(AF_INET, SOCK_STREAM)
# Link server
tcp_client_socket.connect(('192.168.3.31', 8080))
tcp_client_socket.send("Test the content sent".encode("u8"))
# Receive the data sent by the other party, with a maximum of 1024 bytes
recvData = tcp_client_socket.recv(1024)
print('The data received is:', recvData.decode('u8'))
# Close socket
tcp_client_socket.close()

🎏 SOCK_RAW socket 🔥

The above two sockets are conventional socket modes. If the third parameter is omitted or zero (IP Protocol), the correct protocol (TCP protocol and UDP protocol) will be automatically selected.

When we specify the socket type as socket SOCK_ When raw raw sockets, the third parameter needs to specify the proto protocol number.

The predefined protocol numbers of python socket library are:

• socket.IPPROTO_TCP: TCP transport protocol with a value of 6
• socket.IPPROTO_UDP: UDP transport protocol with a value of 17
• socket.IPPROTO_ICMP: ICMP Protocol, with a value of 1
• socket.IPPROTO_IP: IP protocol with a value of 0
• socket.IPPROTO_RAW: the IP header can be built to build a lower layer protocol. The value is 1

You can also obtain the protocol number constant through the protocol name:

import socket

print(socket.IPPROTO_ICMP, socket.getprotobyname("icmp"),
      socket.IPPROTO_ICMP == socket.getprotobyname("icmp"))

1 1 True

You can see that the agreement number can be obtained in two ways.

Through the raw socket, we can use ICMP or lower protocol to communicate, so as to realize higher-level functions.

We need to use ICMP Protocol for network communication, so we can use SOCK_RAW socket:

icmp_socket = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_ICMP)

🔏 Other common methods of socket modules and objects 🔥

Other common methods of socket module:

socket.gethostbyname: converts the host name to IPv4 address format. The IPv4 address is returned as a string

socket.gethostname: returns a string containing the hostname of the machine currently executing by the Python interpreter

socket.gethostbyaddr: get the host name according to the IP address

socket.getprotobyname: convert Internet protocol name to protocol number constant

On machines where the host byte order is different from the network byte order, use the following method to convert:

	Convert network order to host byte order	Convert host order to network byte order
32-bit positive integer 4-byte swap operation	socket.ntohl	socket.htonl
16 bit positive integer 2-byte swap operation	socket.ntohs	socket.htons

On machines where the host byte order is the same as the network byte order, there is no operation to perform the above method.

socket.inet_aton: package the IPv4 address in string format into a 32-bit 4-byte byte object

Method 1 to obtain the local ip address: first obtain the local host name, and then obtain the ip address through the host name

import socket

ip = socket.gethostbyname(socket.gethostname())
print(ip)

192.168.3.31

Obtain the IP addresses of all network cards in this computer:

ips = socket.gethostbyname_ex(socket.gethostname())[-1]
print(ips)

['192.168.3.31']

⚠️ Note: if the host name is not set correctly, the local ip address may not be obtained.

Common functions of socket object:

• s.getpeername(): returns the remote address of the connection socket. The return value is usually a tuple (ipaddr,port)
• s.getsockname(): returns the address of the socket itself. Usually a tuple (ipaddr,port)
• s. Set socket opt (level, optname, value): sets the value of the given socket option.
• s.getsockopt(level,optname[.buflen]): returns the value of the socket option.
• s. Set timeout (timeout): sets the timeout of socket operation. Timeout is a floating-point number in seconds. A value of None indicates that there is no timeout. In general, timeout periods should be set when a socket is first created because they may be used for connection operations (such as connect)
• s.gettimeout(): returns the value of the current timeout period in seconds. If no timeout period is set, it returns None
• s.fileno(): returns the file descriptor of the socket
• s.setblocking(flag): if the flag is 0, set the socket to non blocking mode, otherwise set the socket to blocking mode (default). In non blocking mode, if the call recv() does not find any data, or the call send() cannot send data immediately, it will cause socket Error exception.
• s.makefile(): creates a file associated with the socket.

Method 2: Send a stateless UDP request to any network address, and then obtain your own address through the socket object to obtain the local address

import socket

def get_local_ip():
    with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as s:
        s.connect(('1.1.1.1', 80))
        ip, port = s.getsockname()
        return ip
# Get native IP
ip = get_local_ip()
print(ip)

192.168.3.31

✅ Even if you cannot connect to the Internet and the destination address cannot be accessed (the message will be lost), you can use this method to obtain the local ip address.

📥 Transformation of struct binary data 🔥

Python provides a struct module to solve the conversion of bytes and other binary data types.

The pack function of struct turns any data type into bytes.

import struct
print(struct.pack('>I', 10240099))

b'\x00\x9c@c'

The first parameter of pack is the processing instruction:

• >: indicates that the byte order is big endian, that is, network order
• 1: Represents a 4-byte unsigned integer
• H:2-byte unsigned integer.

The number of subsequent parameter bytes shall be consistent with the processing instruction.
unpack converts bytes into the corresponding data type:

>>> struct.unpack('>IH', b'\xf0\xf0\xf0\xf0\x80\x80')
(4042322160, 32896)

For the data types defined by the struct module, please refer to the official Python documentation:

https://docs.python.org/zh-cn/3/library/struct.html#format-characters

format	Type C	Python type	Standard size	notes
x	Fill byte	nothing
c	char	Byte string with length of 1	1
b	signed char	integer	1	(1), (2)
B	unsigned char	integer	1	-2
?	_Bool	bool	1	-1
h	short	integer	2	-2
H	unsigned short	integer	2	-2
i	int	integer	4	-2
I	unsigned int	integer	4	-2
l	long	integer	4	-2
L	unsigned long	integer	4	-2
q	long long	integer	8	-2
Q	unsigned long long	integer	8	-2
n	ssize_t	integer		-3
N	size_t	integer		-3
e	-6	Floating point number	2	-4
f	float	Floating point number	4	-4
d	double	Floating point number	8	-4
s	char []	Byte string
p	char []	Byte string
P	void *	integer		-5

📇 How Ping works

ping works based on ICMP Protocol. The full name of ICMP is Internet Control Message Protocol, that is, Internet Control Message Protocol. The ICMP message sent by ping actually realizes the control in the form of reconnaissance network state, feeds back the network state, and adjusts the transmission strategy to control the whole situation.

The main functions of ICMP include: * * confirm whether the IP packet is successfully delivered to the target address, report the reason why the IP packet is discarded during sending, and improve network settings** ICMP Protocol is mainly responsible for notifying the reason why an IP packet fails to reach the target address in IP communication.

📂 ICMP message format 🔥

The ICMP message sent by the Ping command is encapsulated in the IP packet. The structure is as follows:

In the above message format, the IP header on the left does not need to be concerned, because the original socket mode of socket will automatically help us encapsulate the IP header, and the ICMP message on the right is the part we need to care about.

⚠️ Note: compared with the native ICMP, the ICMP message sent by the Ping command has two more fields: identifier and serial number.

There are two types of ICMP messages:

1. Query message type: query message for diagnosis
2. Error message type: an error message notifying the cause of the error

However, the PING we use only needs to use the echo response and echo request in the query message type.

Common ICMP types include:

🍋 ICMP query message type 🔥

Loopback message: 0 indicates loopback response and 8 indicates loopback request. A message used between hosts or routers for communication to judge whether the transmitted data packet has successfully arrived at the opposite end.

The ping command is implemented through the echo message of ICMP Protocol:

The sending end host sends a loopback request (ICMP Echo Request Message, type 8) to the receiving end host. As long as the loopback response (ICMP Echo Reply Message, type 0) returned by the receiving end is normally received, it means that the sending end host can reach the receiving end host.

📺 ICMP error message type 🔥

For the error message type, it will not be used in this coding. There is no need to go deep into it. Just have a simple understanding.

ICMP common error message:

• Target unreachable message - type 3
• Origin suppression message - type 4
• Redirect message - type 5
• Timeout message - type 11

Destination Unreachable Message:

When the IP router cannot send the IP packet to the destination address, it will return an ICMP message that the destination is unreachable to the host at the sending end, and the specific reason for the unreachable will be displayed in the message. The reason is recorded in the code field of the ICMP packet header.

Therefore, according to the specific message of ICMP unreachable, the sending host can understand the specific reason for the unreachable transmission.

The reasons why the goal cannot be reached are:

• The network unreachable code is 0
• Host unreachable code is 1
• Protocol unreachable code is 2
• Port unreachable code is 3
• Sharding is required, but the non sharding bit code is set to 4

ICMP source queue message:

The purpose of ICMP origin suppression message is to alleviate the problem of network congestion. When the router sends data to the low-speed line, the cache of its transmission queue becomes zero and cannot be sent out, an ICMP origin suppression message can be sent to the source address of the IP packet.

However, the host receiving this ICMP message may not really increase the transmission interval of IP packets, and may also cause unfair network communication, so it is generally not used.

ICMP Redirect Message:

When the router holds better routing information and finds that the sending host uses a non optimal path to send data, the router will return an ICMP redirection message to the host. This message contains the most appropriate routing information and source data. The sender can send it to another closer router next time.

ICMP Time Exceeded Message:

An 8-bit field in the IP packet is called TTL (Time To Live). Its value will decrease by 1 every time it passes through the router, and the IP packet will be discarded until it decreases to 0.

At this time, the IP router will send an ICMP timeout message to the sending host and notify that the packet has been discarded. The main purpose of setting IP packet life cycle is to avoid IP packets being forwarded endlessly on the network when routing control encounters problems and cyclic conditions.

You can also control the arrival range of packets by setting a smaller TTL value.

📌 Socket raw socket implements PING command

After learning so much basic network knowledge, what are we ultimately for? Is to be able to implement the PING command. There is still a lot of relevant network knowledge, but it doesn't matter much for us to realize the PING command, so we won't go deep into it for the time being.

Let's start from the actual combat, debug step by step, and continue to dig into the implementation principle of PING command.

First, we create the original socket link of ICMP Protocol:

import socket

icmp_socket = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_ICMP)

🌎 Send echo request 🔥

Then you need to send a loopback request to the target. The structure is as follows:

Next, start to organize message data (for serial number, we can decide the value to be sent):

import os
import time
import struct

# The verification needs to be calculated later. Set it to 0 here
ICMP_ECHO_REQUEST, code, checksum, identifier, serial_num = 8, 0, 0, os.getpid() & 0xFFFF, 0
# Initial packaging ICMP head
header = struct.pack("bbHHh", ICMP_ECHO_REQUEST, code,
                     checksum, identifier, serial_num)
# Package option data, including the current timestamp, which is supplemented to 192 bits with Q
data = struct.pack("d", time.time()).ljust(192, b"Q")

The rules for calculating checksum have been written in code here. You can directly look at the code:

def calc_checksum(src_bytes):
    """For calculation ICMP Message checksum"""
    total = 0
    max_count = len(src_bytes)
    count = 0
    while count < max_count:
        val = src_bytes[count + 1]*256 + src_bytes[count]
        total = total + val
        total = total & 0xffffffff
        count = count + 2

    if max_count < len(src_bytes):
        total = total + ord(src_bytes[len(src_bytes) - 1])
        total = total & 0xffffffff

    total = (total >> 16) + (total & 0xffff)
    total = total + (total >> 16)
    answer = ~total
    answer = answer & 0xffff
    answer = answer >> 8 | (answer << 8 & 0xff00)
    return socket.htons(answer)

⚠️ Note: the final return is through socket Htons method converts data from host sequence to network sequence.

Then you can calculate the checksum and repackage the header:

checksum = calc_checksum(header + data)
header = struct.pack("bbHHh", ICMP_ECHO_REQUEST, code,
                     checksum, identifier, serial_num)

Then you can send:

# Send to the target address. ICMP Protocol has no concept of port. The port can be filled in freely
target_addr = "192.168.3.31"
icmp_socket.sendto(header + data, (target_addr, 1))

⚠️ Note: although it is sent to port 1, it can be sent to any port.

🌌 Receive loopback response 🔥

The echo response is consistent with the echo request structure:

After sending the message, we can receive the corresponding echo:

# Receive loopback request
recv_packet, addr = icmp_socket.recvfrom(1024)
# The first 20 bytes are the ip header of the ip protocol
icmp_header = recv_packet[20:28]
data = recv_packet[28:]
ICMP_Echo_Reply, code, checksum, identifier, serial_num = struct.unpack(
    "bbHHh", icmp_header
)
time_sent, = struct.unpack("d", data[:struct.calcsize("d")])

⚠️ Note: the loopback request we received contains the IP headers of the top 20.

The sending time of this packet (the time written when it was sent) can be parsed from the option data.

🎉 Improve the development of ping command 🔥

Although the standard PING command is implemented with the above protocol rules, we do not need to fully follow the above specifications. For example, the identifier can send any 16 bit value, the sequence number can start from any value, and the 192 bit space of option data can also be used to store any data.

When receiving the echo response, we need to check the packet identifier to determine that it is the packet sent by ourselves.

Finally, the following methods are encapsulated:

import struct
import time
import os
import socket
import select


def calc_checksum(src_bytes):
    """For calculation ICMP Message checksum"""
    total = 0
    max_count = len(src_bytes)
    count = 0
    while count < max_count:
        val = src_bytes[count + 1]*256 + src_bytes[count]
        total = total + val
        total = total & 0xffffffff
        count = count + 2

    if max_count < len(src_bytes):
        total = total + ord(src_bytes[len(src_bytes) - 1])
        total = total & 0xffffffff

    total = (total >> 16) + (total & 0xffff)
    total = total + (total >> 16)
    answer = ~total
    answer = answer & 0xffff
    answer = answer >> 8 | (answer << 8 & 0xff00)
    return socket.htons(answer)


def sent_ping(icmp_socket, target_addr, identifier=os.getpid() & 0xFFFF,
              serial_num=0, data=None):
    # The verification needs to be calculated later. Set it to 0 here
    ICMP_ECHO_REQUEST, code, checksum = 8, 0, 0
    # Initial packaging ICMP head
    header = struct.pack("bbHHh", ICMP_ECHO_REQUEST, code,
                         checksum, identifier, serial_num)
    # Package option data
    if data:
        data = data.ljust(192, b"Q")
    else:
        data = struct.pack("d", time.time()).ljust(192, b"Q")
    checksum = calc_checksum(header + data)
    header = struct.pack("bbHHh", ICMP_ECHO_REQUEST, code,
                         checksum, identifier, serial_num)
    # Send to the target address. ICMP Protocol has no concept of port. The port can be filled in freely
    icmp_socket.sendto(header + data, (target_addr, 1))


def receive_pong(icmp_socket, identifier=os.getpid() & 0xFFFF, serial_num=0, timeout=2):
    icmp_socket.settimeout(timeout)
    time_remaining = timeout
    while True:
        start_time = time.time()
        # Receive loopback request
        recv_packet, (ip, port) = icmp_socket.recvfrom(1024)
        time_received = time.time()
        time_spent = time_received-start_time
        # The first 20 bytes are the ip header of the ip protocol
        icmp_header = recv_packet[20:28]
        data = recv_packet[28:]
        ICMP_Echo_Reply, code, checksum, identifier_reciver, serial_num_reciver = struct.unpack(
            "bbHHh", icmp_header
        )
        if identifier_reciver != identifier or serial_num != serial_num_reciver:
            # If the package is not sent by yourself, it will be ignored
            time_remaining -= time_spent
            if time_remaining <= 0:
                raise socket.timeout
            continue
        time_sent, = struct.unpack("d", data[:struct.calcsize("d")])
        return int((time_received - time_sent)*1000), ip

192.168. 3.31 is the LAN IP address of my current machine. Test it:

icmp_socket = socket.socket(
    socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_ICMP)
ip = '192.168.3.31'
sent_ping(icmp_socket, ip)
try:
    delay, ip_received = receive_pong(icmp_socket, timeout=2)
    print(f"delay:{delay}ms，other party ip:{ip_received}")
except socket.timeout as e:
    print("overtime")

delay:0ms，other party ip:192.168.3.31

Then batch ping all IP addresses of the current network segment:

def get_local_ip():
    with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as s:
        s.connect(('1.1.1.1', 80))
        ip, port = s.getsockname()
        return ip


icmp_socket = socket.socket(
    socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_ICMP)

local_ip = get_local_ip()
net_segment = local_ip[:local_ip.rfind(".")]
ips = []
for i in range(1, 255):
    ip = f"{net_segment}.{i}"
    sent_ping(icmp_socket, ip)
    print("ping", ip, end=" ")
    try:
        delay, ip_received = receive_pong(icmp_socket, timeout=0.1)
        print(f"delay:{delay}ms，other party ip:{ip_received}")
        ips.append(ip)
    except socket.timeout as e:
        print("overtime")
print(ips)
icmp_socket.close()

When the timeout is 0.1 seconds, the total time is 30 seconds:

When the timeout is set to 0.01 seconds, the total time is 2.59 seconds.

🔔 Get online devices of current network segment with arp table 🔥

**How to get the current online devices as soon as possible** After testing, it is found that the arp table can automatically delete the corresponding entries for machines that cannot be pinged after being pinged. Then idea 1 is to quickly send a loopback request to the whole network segment without waiting for a loopback response, and then check the arp table 2 seconds later to see the latest online devices.

Implementation idea 1:

import struct
import time
import os
import re
import socket
import pandas as pd


def calc_checksum(src_bytes):
    """For calculation ICMP Message checksum"""
    total = 0
    max_count = len(src_bytes)
    count = 0
    while count < max_count:
        val = src_bytes[count + 1]*256 + src_bytes[count]
        total = total + val
        total = total & 0xffffffff
        count = count + 2

    if max_count < len(src_bytes):
        total = total + ord(src_bytes[len(src_bytes) - 1])
        total = total & 0xffffffff

    total = (total >> 16) + (total & 0xffff)
    total = total + (total >> 16)
    answer = ~total
    answer = answer & 0xffff
    answer = answer >> 8 | (answer << 8 & 0xff00)
    return socket.htons(answer)


def sent_ping(icmp_socket, target_addr, identifier=os.getpid() & 0xFFFF,
              serial_num=0, data=None):
    # The verification needs to be calculated later. Set it to 0 here
    ICMP_ECHO_REQUEST, code, checksum = 8, 0, 0
    # Initial packaging ICMP head
    header = struct.pack("bbHHh", ICMP_ECHO_REQUEST, code,
                         checksum, identifier, serial_num)
    # Package option data
    if data:
        data = data.ljust(192, b"Q")
    else:
        data = struct.pack("d", time.time()).ljust(192, b"Q")
    checksum = calc_checksum(header + data)
    header = struct.pack("bbHHh", ICMP_ECHO_REQUEST, code,
                         checksum, identifier, serial_num)
    # Send to the target address. ICMP Protocol has no concept of port. The port can be filled in freely
    icmp_socket.sendto(header + data, (target_addr, 1))

def get_arp_ip_mac():
    header = None
    with os.popen("arp -a") as res:
        for line in res:
            line = line.strip()
            if not line or line.startswith("Interface"):
                continue
            if header is None:
                header = re.split(" {2,}", line.strip())
                break
        df = pd.read_csv(res, sep=" {2,}",
                         names=header, header=0, engine='python')
    return df


def ping_net_segment_all(net_segment):
    with socket.socket(
            socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_ICMP) as icmp_socket:
        for i in range(1, 255):
            ip = f"{net_segment}.{i}"
            sent_ping(icmp_socket, ip)


net_segment = "192.168.3"
ping_net_segment_all(net_segment)
# Wait for the arrival of the echo response, which is expected to be within 1 second
time.sleep(1)
# Read the latest arp table
df = get_arp_ip_mac()
df

So we get the online device list of the current network segment:

📇 Dual thread gets the online device of the specified network segment 🔥

However, there is a defect in using the arp table. You can only view the of the current network segment. It seems that online devices across network segments cannot be seen. After analysis, the desktop I use is connected to network segment 3 through wired connection, while the mobile phone is connected to network segment 2 through WiFi, so it is meaningful to be able to analyze the online devices of network segment 2 devices.

Idea 2: two threads are used. One thread is dedicated to sending a loopback request and the other thread is dedicated to receiving a loopback response. The IP address can be obtained through the loopback response, so the IP address of the currently online device of the specified network segment can be obtained.

Get the online device list first:

from concurrent.futures import ThreadPoolExecutor
import _thread
import struct
import time
import os
import re
import socket
import pandas as pd


def calc_checksum(src_bytes):
    """For calculation ICMP Message checksum"""
    total = 0
    max_count = len(src_bytes)
    count = 0
    while count < max_count:
        val = src_bytes[count + 1]*256 + src_bytes[count]
        total = total + val
        total = total & 0xffffffff
        count = count + 2

    if max_count < len(src_bytes):
        total = total + ord(src_bytes[len(src_bytes) - 1])
        total = total & 0xffffffff

    total = (total >> 16) + (total & 0xffff)
    total = total + (total >> 16)
    answer = ~total
    answer = answer & 0xffff
    answer = answer >> 8 | (answer << 8 & 0xff00)
    return socket.htons(answer)


def sent_ping(icmp_socket, target_addr, identifier=os.getpid() & 0xFFFF,
              serial_num=0, data=None):
    # The verification needs to be calculated later. Set it to 0 here
    ICMP_ECHO_REQUEST, code, checksum = 8, 0, 0
    # Initial packaging ICMP head
    header = struct.pack("bbHHh", ICMP_ECHO_REQUEST, code,
                         checksum, identifier, serial_num)
    # Package option data
    if data:
        data = data.ljust(192, b"Q")
    else:
        data = struct.pack("d", time.time()).ljust(192, b"Q")
    checksum = calc_checksum(header + data)
    header = struct.pack("bbHHh", ICMP_ECHO_REQUEST, code,
                         checksum, identifier, serial_num)
    # Send to the target address. ICMP Protocol has no concept of port. The port can be filled in freely
    icmp_socket.sendto(header + data, (target_addr, 1))


def receive_pong(icmp_socket, net_segment, timeout=2):
    icmp_socket.settimeout(timeout)
    ips = set()
    while True:
        start_time = time.time()
        try:
            recv_packet, (ip, port) = icmp_socket.recvfrom(1024)
            if ip.startswith(net_segment):
                ips.add(ip)
        except socket.timeout as e:
            break
    return ips


def ping_net_segment_all(icmp_socket, net_segment):
    for i in range(1, 255):
        ip = f"{net_segment}.{i}"
        sent_ping(icmp_socket, ip)


icmp_socket = socket.socket(
    socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_ICMP)
with ThreadPoolExecutor() as p:
    p.submit(ping_net_segment_all, icmp_socket, "192.168.2")
    future = p.submit(receive_pong, icmp_socket, "192.168.2", 3)
    ips = future.result()

ips

As a result, my mobile ip is 192.168 2.122, it is successfully detected after operation:

{'192.168.2.1',
 '192.168.2.122',
 '192.168.2.17',
 '192.168.2.18',
 '192.168.2.19',
 '192.168.2.20',
 '192.168.2.21',
 '192.168.2.22',
 '192.168.2.23',
 '192.168.2.49'}

After turning off the mobile WiFi, run it again and see the offline of the IP smoothly.

📟 Complete the fishing artifact of the BOSS 🔥

When the update time has been shortened to less than 5 seconds, we can PING the specified network segment and finally complete the online and offline functions of the analysis equipment, so as to achieve the ultimate goal and complete the fishing artifact from the BOSS.

from concurrent.futures import ThreadPoolExecutor
import _thread
import struct
import time
import os
import re
import socket
import pandas as pd


def calc_checksum(src_bytes):
    """For calculation ICMP Message checksum"""
    total = 0
    max_count = len(src_bytes)
    count = 0
    while count < max_count:
        val = src_bytes[count + 1]*256 + src_bytes[count]
        total = total + val
        total = total & 0xffffffff
        count = count + 2

    if max_count < len(src_bytes):
        total = total + ord(src_bytes[len(src_bytes) - 1])
        total = total & 0xffffffff

    total = (total >> 16) + (total & 0xffff)
    total = total + (total >> 16)
    answer = ~total
    answer = answer & 0xffff
    answer = answer >> 8 | (answer << 8 & 0xff00)
    return socket.htons(answer)


def sent_ping(icmp_socket, target_addr, identifier=os.getpid() & 0xFFFF,
              serial_num=0, data=None):
    # The verification needs to be calculated later. Set it to 0 here
    ICMP_ECHO_REQUEST, code, checksum = 8, 0, 0
    # Initial packaging ICMP head
    header = struct.pack("bbHHh", ICMP_ECHO_REQUEST, code,
                         checksum, identifier, serial_num)
    # Package option data
    if data:
        data = data.ljust(192, b"Q")
    else:
        data = struct.pack("d", time.time()).ljust(192, b"Q")
    checksum = calc_checksum(header + data)
    header = struct.pack("bbHHh", ICMP_ECHO_REQUEST, code,
                         checksum, identifier, serial_num)
    # Send to the target address. ICMP Protocol has no concept of port. The port can be filled in freely
    icmp_socket.sendto(header + data, (target_addr, 1))


def receive_pong(icmp_socket, net_segment, timeout=2):
    icmp_socket.settimeout(timeout)
    ips = set()
    while True:
        start_time = time.time()
        try:
            recv_packet, (ip, port) = icmp_socket.recvfrom(1024)
            if ip.startswith(net_segment):
                ips.add(ip)
        except socket.timeout as e:
            break
    return ips


def ping_net_segment_all(icmp_socket, net_segment):
    for i in range(1, 255):
        ip = f"{net_segment}.{i}"
        sent_ping(icmp_socket, ip)


last = None
while 1:
    icmp_socket = socket.socket(
        socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_ICMP)
    with ThreadPoolExecutor() as p:
        p.submit(ping_net_segment_all, icmp_socket, "192.168.2")
        future = p.submit(receive_pong, icmp_socket, "192.168.2")
        ips = future.result()
    if last is None:
        print("Currently online devices:", ips)
    if last:
        up = ips-last
        if up:
            print("\r New online equipment:", up, end=" "*100)
        down = last-ips
        if down:
            print("\r Newly offline equipment:", down, end=" "*100)
    last = ips
    time.sleep(3)

Examples of results:

Currently online devices: {'192.168.2.122', '192.168.2.18', '192.168.2.20', '192.168.2.1', '192.168.2.23', '192.168.2.49', '192.168.2.21', '192.168.2.17', '192.168.2.22', '192.168.2.19'}
Newly offline equipment: {'192.168.2.122'}  

After testing, manually turning off or on the mobile WiFi can successfully see the printed information of the device IP. Although this method cannot obtain the MAC address, after testing, the same machine will be assigned the same IP, which meets the requirements in my current network. You only need to know the IP connected to the boss's mobile phone. Or observe which IP goes offline after the boss leaves, and specifically monitor this IP.

The safer way is to be extra vigilant every time you see a new IP online. If you are a win10 system, you can use the following methods to realize system notification:

from win10toast import ToastNotifier

toaster = ToastNotifier()
toaster.show_toast("Notice title", "Notice content!", duration=10)

The above three parameters are notification title, notification content and notification duration respectively. For fishing, the duration can be increased, and then the notification can be manually closed and installed through pip install win10toast.

☀️ summary

Finally made this fishing artifact, but although I said it with interest, no one really plans to use this code to deal with the boss ⁉️ No, No ⁉️

I really intend to make children's shoes for fishing artifact. I personally recommend setting up a webcam and writing a character image recognition code. When I find someone coming in, I will automatically remind them, so that I can fish more safely. If the boss comes without wifi, it's a bit of a pit.

Developing fishing artifact is not the purpose of this paper. Learning network knowledge and realizing network protocol independently is the real purpose of this paper. In order to conceive this article, I have been thinking hard for several days and nights. Xiaoming is here to ask you for a 3-company online, OK? 💖

I'm Xiao Ming. I'll see you next time. Don't forget to light up the little red heart for three times