socket network programming based on TCP protocol
1, What is socket
1. Introduction:
After understanding the osI seven layer protocol, we can see that there is a socket abstraction layer between the application layer and the transport layer. The abstraction layer here does not exist in the osI seven layer protocol. The socket abstraction layer here is an interface for the application layer to communicate through the network after passing through all the following layers
2. What is socket?
As shown in the figure:
- Socket is an intermediate software abstraction layer for the communication between application layer and TCP/IP protocol family. It is a group of interfaces. In the design mode, socket is actually a facade mode. It hides the complex TCP/IP protocol family behind the socket interface. For users, a set of simple interfaces is all, allowing the socket to organize data to comply with the specified protocol.
- Therefore, we do not need to deeply understand the tcp/udp protocol. The socket has been encapsulated for us. We only need to follow the rules of socket to program. The written program naturally follows the tcp/udp standard.
3. Benefits of socket encapsulation:
- For application developers, they only need to pay attention to the relevant things in the application. How do you want to deal with and encapsulate your data in the application layer, What kind of protocol is customized in the application layer, You can do whatever you want. You just need to hand over the data processed by the application layer to socket，It will help you deal with the transport layer and the network layer, To data link layer, What needs to be done to the physical layer.
4. What is the purpose of studying socket abstraction level?
- socket The abstraction layer is not to help you write your application, but to send the data written by your application based on the network protocol
5. Note:
- Note: some people refer to socket as ip+port. ip is used to identify the location of a host in the Internet, while port is used to identify an application on the machine. The ip address is configured on the network card, while port is enabled by the application. The binding of ip and port identifies a unique application in the Internet, The pid of the program is the identification of different processes or threads on the same machine.
2, Development history and classification of socket
Socket originated from the version of Unix at the University of California, Berkeley in the 1970s, which is called BSD Unix. Therefore, socket is sometimes called "Berkeley socket" or "BSD socket". Initially, sockets were designed to communicate between multiple applications on the same host. This is also known as interprocess communication, or IPC. There are two kinds of sockets (or two races), which are file based and network-based.
1. Socket family based on file type
Socket family name: AF_UNIX
- unix is all about files. File based sockets call the underlying file system to get data. Two socket processes run on the same machine and can communicate indirectly by accessing the same file system
2. Socket family based on network type
Socket family name: AF_INET
(in addition, AF_INET6 is used for ipv6, and there are other address families. However, they are either only used for a certain platform, or they have been abandoned, or they are rarely used, or they are not implemented at all. Among all address families, AF_INET is the most widely used. python supports many address families, but because we only care about network programming, so Most of the time we only use AF_INET)
3. Transmission protocol type
Streaming socket: SOCK_STREAM
- Stream sockets are used to provide connection oriented and reliable data transmission services. The service will ensure that the data can be sent without error and repetition, and received in sequence. The reason why streaming socket can realize reliable data service is that it uses transmission control protocol, namely TCP protocol
Datagram socket: SOCK_DGRAM
- Datagram sockets provide a connectionless service. This service can not guarantee the reliability of data transmission. The data may be lost or duplicated in the transmission process, and the data can not be received sequentially. Datagram socket uses UDP protocol for data transmission. Since datagram socket can not guarantee the reliability of data transmission, it is necessary to deal with the possible data loss in the program
Raw socket: SOCK_RAW
- The difference between the original socket and the standard socket (standard socket refers to the stream socket and datagram socket introduced earlier) is that the original socket can read and write IP packets that the kernel does not process, while the stream socket can only read the data of TCP protocol, and the datagram socket can only read the data of UDP protocol. Therefore, if you want to access the data sent by other protocols, you must use the original socket
4. Summary
Early sockets were not used for network communication. They were generally used to handle the communication between two processes on a computer
We know that the computer starts two processes and applies for the memory space. The memory space of the two processes is isolated from each other and physically isolated in order to ensure the safety of data, so the data between processes cannot be interactive
However, the hard disk is common to all programs, so you can write the data of one process to the hard disk first and let another process take it from the hard disk, so as to realize the data interaction between processes
3, Workflow of socket
The connection process between client and server can be divided into four steps, as follows:
1. The server binds IP+Port and establishes listening
- After initializing the Socket dynamic library, the client creates a Socket, then specifies the address of the client Socket, binds the Socket circularly until it succeeds, and then starts to establish monitoring. At this time, the client is in the waiting state and monitors the network state in real time
2. The client sends a request to the server
- The client's Socket makes a connection request to the server. At this time, the client describes the Socket it wants to connect to, points out the relevant properties of the Socket to be connected, and then makes a request to the server's Socket
3. Connection request confirmation and establishment
- When the server-side socket monitors the connection request from the client, it immediately responds to the request and establishes a new process, and then feeds back the description of the server-side socket to the client. After the client confirms, the connection is established successfully
4. Send and receive data after the connection is established successfully
- Then the client and the server can communicate with each other and transmit data. At this time, the socket on the server continues to wait to listen for requests from other clients
5. Summary:
client:
socket()
The abstraction layer in the client application socket Specifies the transport layer protocol of the next layer to deal with directly.(Here with TCP/UPD take as an example)
connect()
Then by getting TCP Server side IP And port, find the unique program that communicates with it on the server host.(Tip: the client does not need to be bound. The port of the client. When accessing the server, the server can get the port of the client. Therefore, the client port does not need to be bound)
read()and write()
After the data is successfully linked with the server, the data can be sent and received.
close()
last, Send a system call to the operating system, close the connection path between the client and the server, and recycle the system resources occupied when establishing the connection,
Server:
socket()
Server designation socket The protocol that deals with the transport layer. The protocol of the transport layer of the next layer.(Here with TCP/UPD take as an example)
bind()
The server needs to be bound Ip And port, and wait for the client to pass this Ip And port, You can find this program on this server that is unique in the world, And communicate with it.
listen()
Before the three handshakes establish the link, the server itself is in the LISHEN At this time, the link of the server is still in the semi connection pool and has not been linked by the client
accept()
It's actually Tcp Establish a link where you shake hands three times. If you have a request, you will go listen Get connections from the semi connection pool of. It is based on a series of services. After one service is finished, the next service can be carried out
read()and write()
After the client is successfully established, Then the server can send and receive data to the client.
close()
last, Send a system call to the operating system, close the connection path between the server and the client, and recycle the system resources occupied when establishing the connection,
6. Python code implementation
import socket # socket_family can be AF_UNIX or AF_INET. socket_type can be SOCK_STREAM or SOCK_DGRAM. protocol is generally not filled in, and the default value is 0 socket.socket(socket_family, socket_type, protocal=0) # Get tcp/ip socket tcpSock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # Get udp/ip socket udpSock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # Because there are too many attributes in the socket module. Here we make an exception and use the 'from module import *' statement. Using 'from socket import *', we will bring all the attributes in the socket module into our namespace, which can greatly shorten our code tcpSock = socket(AF_INET, SOCK_STREAM)
4, Common functions (Methods) of socket module
1. Socket instantiation (get a socket object)
- Format: socket (family, type, protocol = 0)
- Three parameters: common protocol group, socket type, and specified protocol (0 by default)
- AF_ INET (default), AF_INET6,AF_UNIX,AF_ROUTE et al
- SOCK_STREAM(TCP type) (default), SOCK_DGRAM(UDP type), sock_ Raw (original type)
- Normally, it is not written. The default is' 0 '. The default protocol and type are used
s=socket.socket() # Equivalent to the following socket.socket(socket.AF_INET,socket.SOCK_STREAM) # Equivalent to above s=socket.socket(socket.AF_INET,socket.SOCK_DGRAM) # If you want to use UDP, you have to write it like this
2. Server socket function
| function | explain |
|---|---|
| s.bind() | Bind the address (host,port) to the socket in AF_ Under INET, the address is expressed in the form of tuple (host,port) |
| s.listen() | Start TCP listening. backlog specifies the maximum number of connections that the operating system can suspend before rejecting the connection. This value is at least 1, and generally 5 is enough |
| s.accept() | Passively accept TCP client connections, (blocking) waiting for the arrival of the connection |
3. Client socket function
| function | explain |
|---|---|
| s.connect() | Actively initialize TCP server connection,. Generally, the format of address is tuple (hostname,port). If there is a connection error, socket is returned Error error |
| s.connect_ex() | The extended version of the connect() function. When an error occurs, it returns an error code instead of throwing an exception |
4. Public purpose socket function
| function | explain |
|---|---|
| s.recv() | Receive TCP data. The data is returned as a string. The buffer (bufsize) specifies the maximum amount of data to be received. flag provides additional information about the message, which can usually be ignored |
| s.send() | Send TCP data and send the data in string to the connected socket. The return value is the number of bytes to be sent, which may be less than the byte size of string (hint: less than when null is entered) |
| s.sendall() | Send TCP data completely. Sends the data in the string to the connected socket, but attempts to send all the data before returning. None is returned successfully, and an exception is thrown if it fails |
| s.recvfrom() | Receive UDP data, similar to recv(), but the return value is (data_bytes,address). Where data_ Bytes is the received data of bytes type, and address is the address of sending data, which is expressed in the form of tuple ('IP ', port) |
| s.sendto() | Send UDP data and send the data to the socket. Address is a tuple in the form of (ipaddr, port) and specifies the remote address. The return value is the number of bytes sent |
| s.close() | Close socket |
| s.getpeername() | Returns the remote address of the connection socket. The return value is usually a tuple (ipaddr,port) |
| s.getsockname() | Returns the socket's own address. Usually a tuple (ipaddr,port) |
| s.setsockopt(level,optname,value) | The value of the given socket option setting. |
| s.getsockopt(level,optname[.buflen]) | Returns the value of the socket option. |
5. Lock oriented socket method
| function | explain |
|---|---|
| s.setblocking(flag) | If the flag is 0, set the socket to the non blocking mode; otherwise, set the socket to the blocking mode (the default). In the non blocking mode, if the call recv() does not find any data, or the call send() cannot send data immediately, it will cause socket Error exception |
| s.settimeout(timeout) | Set the timeout of socket operation. Timeout is a floating-point number in seconds. A value of None indicates that there is no timeout. In general, timeout periods should be set when a socket is first created, as they may be used for connection operations (such as connect()) |
| s.gettimeout() | Returns the value of the current timeout period in seconds. If the timeout period is not set, it returns None |
6. File oriented socket function
| function | explain |
|---|---|
| s.fileno() | Returns the file descriptor of the socket |
| s.makefile() | Create a file associated with the socket |
5, Socket based on TCP
Note: TCP is based on connection communication, so you must start the server first, and then start the client
1. Analog telephone simple communication connection
-
TCP server
"""
from socket import *
phone = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
First parameter: Specifies the type of work in the socket family.
AF_INET: Specifies the socket type for network-based communication.(supplement: AF Full name Address Family)
Second parameter: What is specified is to let socket The type of protocol that this abstraction layer deals with the next transport layer.
SOCK_STREAM: This refers to a streaming protocol, which refers to TCP agreement.
phone.bind(('IP', PORT))
1) The first parameter specifies IP，String format.
2) The second parameter specifies the port,(Port range 0~65535. Where 0~1023 Are recognized ports, Also known as familiar port. It's best not to use it, Otherwise, there will be port conflict.)
supplement:
- Server bound Ip And port, Among them Ip The address is in your own LAN Ip address. If you want to be accessed by hosts in other local area networks, you have to apply for an address in the public network and bind the public network address to the address of this server.
- 0.0.0.0: such IP The address can be accessed by any machine, For this machine, it is a "shelter", and all "three noes" who do not know are sent in.(Detailed website: https://baike.baidu.com/item/0.0.0.0/7900621?fr=aladdin)
- 127.0.0.1: This address is the loopback test address, which can only be accessed locally, Even in the same LAN, another host cannot access it.
- stay bind There is a command before the operation setsockopt Yes, reuse your port(For testing)
phone.listen(5)
be careful!!! Here is the connection request, It is not the connection after successful establishment.
1) Here, the parameter specifies the size of the semi connection pool(int type). My designation here is 5, Here, connection requests can be made by 6 clients, When the seventh client connects, it cannot successfully obtain the link request in the semi connection pool.
2) significance: Using this semi connection pool, you can reasonably control the number of connection requests that your server is accessed by the client, and then control the client's access to the server's resources, Prevent the server from accessing too many resources by the client, causing the server to die.
3) performance: At present, when there is a client and the link is successful, The server is providing services for this client. If five clients initiate link requests, these five clients will wait(Because concurrency is not implemented here)，Wait for the last client to disconnect, Good communication with the server accept Establish connection. (be careful!!: The client that the server is providing a communication cycle for the client is also counted as a link request, so it is a total of 6 times. The semi connection pool here can only support 6 clients to initiate connection requests.)
conn, client_addr = phone.accept()
1) accept: Here is waiting for the client to send syn Link request, If there is no client to link, it will be blocked here, In a state of waiting.
2) conn: This is a socket object belonging to the client, which communicates with the client by getting the object in this suite.
3) client_addr: The return value is in the form of a tuple
- The first parameter in the tuple is an object that establishes a two-way link path between the server and the client.
- The second parameter in tuple, I got the client Ip And ports, both of which are also included in the same group.
be careful: If the client and server are tested and run on the same host, the server will get it client_addr Medium Ip And port. The port must be different from the server port, because the port on the same host is unique.
data = conn.recv(1024): The maximum amount of received data specified here is 1024 bytes(Bytes). Returns a bytes type. (be careful: This number cannot exceed the byte receiving size of memory. Data processing is transferred in memory, so it cannot exceed.)
conn.send(bytes Type of data): Send is specified here bytes Type of data, to conn This connection path to the client sends data.
conn.close(): When the client sends FIN When the request ends, the link path of the client should be closed here, At the same time, it also reclaims the connection resources with the client system previously applied on the server.(Operation required)
phone.close(): Here is the system process of closing the server(Optional operation: Because the server should not be shut down, it should be running all the time.)
supplement: Here, the server is closed and bound IP If the and port are closed, the operating system needs to recycle this resource. It takes a certain time to recycle this resource, so if you need to run the above program again bind(), The port needs to be reassigned to allow the operating system to make time for the recycling of system resources applied for by the last specified port.
"""
import socket
🧇1.Buy a mobile phone(establish socket object)
phone = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# The first parameter is based on the network socket, and the second is the type: tcp is called the streaming protocol, and udp is called the datagram protocol (SOCK_DGRAM)
🍛2.Card insertion/Binding card (binding address and port number)
phone.bind(('127.0.0.1', 8060))
# A tuple, which is bound with a local loopback address. It can only communicate with the local machine. It is generally used in the test phase
🦪3.Power on (in monitoring state)
phone.listen(5)
# 5 is the set semi connection pool, which limits the number of requests. It can only communicate with one client at the same time, and others are suspended
🥘4.Waiting for a phone connection (waiting for the client's request to connect)
print('Waiting for connection...')
conn, client_addr = phone.accept()
# (conn: the link established by three handshakes, client_addr: the IP and port of the client)
print(conn)
print(client_addr)
🍟5.Start communication: receive/Send a message
data = conn.recv(1024) # Maximum number of bytes accepted 1024
print('Data from client:', data)
conn.send(data.upper()) # Return data
🍿6.Hang up the phone connection (close) TCP Connection)
conn.close()
🍟7.Shutdown (server shutdown)
phone.close()
-
TCP Client
"""
from socket import *
phone = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
phone.connect(('IP', PORT)): The server you need to connect to is specified here IP And port.
phone.send(bytes Type data): Specify here bytes Type data, to pipe server 1 in two-way path --> Data is transmitted in the pipeline of client 1.
data = phone.recv(1024)
phone.close(): Here is the message sent from the client to the server FIN End request, At the same time, it also reclaims the connection resources of the system on the client.(Required operation)
"""
import socket
🍟1.Buy a mobile phone
phone = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print(phone)
🍟2.Dial the phone (the client address will change, and there is no need to bind the address)
phone.connect(('127.0.0.1', 8060))
# Specify the server IP and port, a tuple. Here is the connection between the client and the server
🍟3.Start communication: send/Receive message, Communication cycle
while True:
msg = input('Please enter a message>>').strip()
if len(msg) == 0:continue #The solution here is that the client sends empty data, but there is no data in the client operating system cache sent to the server, so the server cannot get the number from the operating system cache of the server, so the server and the client are in a recv blocked state at the same time,
phone.send(msg.encode('utf-8'))
data = phone.recv(1024) # Maximum accepted bytes 1024
print(data.decode("utf-8"))
🍟4.Close client
phone.close()
2. Problem: restart the server and report an error Address already in uer. Solution
🎂stay bind()Add a line before the function“ socket"to configure phone.setsockopt(SOL_SOCKET,SO_REUSEADDR,1) # Indicates reuse of ip and ports 🍖tcp Is a simple chat program
6, Application examples
TCP is a simple chat program
Requirements: implement a simple chat program, one time, and one connection can be disconnected before connecting to the next.
- Server
import socket
phone=socket.socket(socket.AF_INET,socket.SOCK_STREAM)
phone.bind(("127.0.0.1",8088))
phone.listen(5)
while True:
print("start recv...")
conn,client_addr=phone.accept()
print(f"Connected to a client{client_addr}")
while True:
try:
data=conn.recv(1024)
if len(data)== 0:break
print(f'Data from customers:\033[1;30;46m{data.decode("utf-8")}\033[0m')
user=input("Please enter the content to send (press q sign out, enter (send)>>>: ")
if user.lower()=="q":break
conn.send(user.encode("utf-8"))
except ConnectionError:
break
conn.close()
- client
import socket
phone=socket.socket(socket.AF_INET,socket.SOCK_STREAM)
phone.connect(("127.0.0.1",8088))
while True:
msg=input("Please enter the content to send (press q Exit:").strip()
if msg.lower()=="q":break
if len(msg)==0 :continue
phone.send(msg.encode("utf-8"))
data=phone.recv(1024)
print(f"Voice message from girlfriend: \033[1;30;46m{data.decode('utf-8')}\033[0m")
phone.close()
7, Summary
1. What kinds of operations cause blocking?
- accept, recv, send: accept, recv is the one that obviously causes the blocking operation
2. The server needs to deal with the problem that the client sends empty data (cause: send can be empty, but recv cannot be empty.)
- Cause a problem: if the client is empty, the server will always be blocked in place. At this time, the client will not receive the data sent by the server and will also be blocked in place at the same time
- Solution: the client should make a judgment so that the input content of the client cannot be empty
- Low level principle: the send operation of the client does not correspond to the recv operation of the server The sending and receiving of client and server are aimed at themselves
3. If the client forcibly terminates the connection, the server will have an exception If it is in windows system, it will throw an exception (ConnectionAbortedError) If it is in linux system, recv will always receive null, thus entering the dead loop
- Originally, there was a normal link between the client and the server. The client did not discuss with the server. The client forcibly disconnected the link, and the conn socket object of the server thought that the invalid link of the client was still normal and was receiving the data of the client based on a normal behavior
-
For example: after the connection between the client and the server based on TCP protocol communication is established, a bridge pier is built between them. If the client blows up the bridge pier, the server will blow up, so we need to provide solutions to keep the server unaffected
-
Solve the exception in linux system: judge whether the received data is empty in the server (Note: the null here is not the null received by the server after the client sends the null data, but an error in the linux system after the client forcibly terminates the link.)
-
Solve the exception in window system: use exception handling mechanism
Supplement!!!: Another situation is that the client disconnects with close() normally, and the server in windows system will also receive empty data
4. After the client forcibly terminates the link, the server has been in the state of providing services all the time
- Problem: this kind of link loop is just a kind of one-to-one service. You can't be distracted. Only after the client of the previous link is disconnected, the connection of the next linked client can be established
- For example: the connection loop is like a, soliciting, and the communication loop is receiving. Only one is needed to solicit, while our receiving needs many to serve many clients. At this time, we need concurrency
- Solution: Based on the communication loop of the client, set a while loop to receive the client link, so as to achieve the purpose of link loop