Network foundation, socket programming It's not popular. For me, it's really an old-fashioned concept. python Network Programming Beginner level chapter, Advanced

1, Realize TCP communication

1. TCP protocol

TCP/IP(Transmission Control Protocol/Internet Protocol), i.e. transmission control protocol / inter network protocol, is a connection oriented (connection oriented), reliable and byte stream based Transport layer communication protocol. Because it is a connection oriented protocol, data is transmitted like water flow, and there will be packet sticking problem.

2. TCP server

A TCP server can connect many clients at the same time. Because it is very convenient and efficient to create multiple goroutines in Go language to realize concurrency, we can create a goroutine for processing every time we establish a link. Of course, it is more recommended to establish a goroutine worker pool.

Processing flow of TCP server program:

1. Listening port
2. Receive client request to establish link
3. Create goroutine processing links.

Implementation code:

// tcp/server/main.go

// TCP server side

// Processing function
func process(conn net.Conn) {
	defer conn.Close() // Close connection
	for {
		reader := bufio.NewReader(conn)
		var buf [128]byte
		n, err := reader.Read(buf[:]) // Read data
		if err != nil {
			fmt.Println("read from client failed, err:", err)
			break
		}
		recvStr := string(buf[:n])
		fmt.Println("received client Data sent from terminal:", recvStr)
		conn.Write([]byte(recvStr)) // send data
	}
}

func main() {
	listen, err := net.Listen("tcp", "127.0.0.1:20000")
	if err != nil {
		fmt.Println("listen failed, err:", err)
		return
	}
	for {
		conn, err := listen.Accept() // Remove a connection handler from the connection pool
		if err != nil {
			fmt.Println("accept failed, err:", err)
			continue
		}
		go process(conn) // Start a goroutine processing connection
	}
}

3. TCP client

The process of TCP communication with a TCP client is as follows:

1. Establish a link with the server
2. Send and receive data
3. Close link

The implementation code is as follows:

// tcp/client/main.go

// client
func main() {
	conn, err := net.Dial("tcp", "127.0.0.1:20000")
	/* defer conn.Close()
	 * conn.close What's the difference between putting it here and putting it below?
	 * A: if the following err is not empty, conn cannot call close, and close will directly report an error, exit the program, and put the 				  You can also print to err below.
	 */
	if err != nil {
		fmt.Println("err :", err)
		return
	}
	defer conn.Close() // Close connection 
	inputReader := bufio.NewReader(os.Stdin)
	for {
		input, _ := inputReader.ReadString('\n') // Read user input
		inputInfo := strings.Trim(input, "\r\n")
		if strings.ToUpper(inputInfo) == "Q" { // If you enter q, exit
			return
		}
		_, err = conn.Write([]byte(inputInfo)) // send data
		if err != nil {
			return
		}
		buf := [512]byte{}
		n, err := conn.Read(buf[:])
		if err != nil {
			fmt.Println("recv failed, err:", err)
			return
		}
		fmt.Println(string(buf[:n]))
	}
}

4. TCP sticky packet

The server code remains unchanged, and the client code is as follows:

// socket_stick/client/main.go

func main() {
	conn, err := net.Dial("tcp", "127.0.0.1:30000")
	if err != nil {
		fmt.Println("dial failed, err", err)
		return
	}
	defer conn.Close()
	for i := 0; i < 20; i++ {
		msg := `Hello, Hello. How are you?`
		conn.Write([]byte(msg))
	}
}

Save the above code and compile them respectively. Start the server first and then the client. You can see the output results of the server as follows:

received client Data sent: Hello, Hello. How are you?Hello, Hello. How are you?Hello, Hello. How are you?Hello, Hello. How are you?Hello, Hello. How are you?
received client Data sent: Hello, Hello. How are you?Hello, Hello. How are you?Hello, Hello. How are you?Hello, Hello. How are you?Hello, Hello. How are you?Hello, Hello. How are you?Hello, Hello. How are you?Hello, Hello. How are you?
received client Data sent: Hello, Hello. How are you?Hello, Hello. How are you?
received client Data sent: Hello, Hello. How are you?Hello, Hello. How are you?Hello, Hello. How are you?
received client Data sent: Hello, Hello. How are you?Hello, Hello. How are you?

The data sent by the client in 10 times is not successfully output 10 times at the server, but multiple pieces of data are "stuck" together.

(1) Why are there sticky bags?

The main reason is that tcp data transfer mode is stream mode, which can receive and send multiple times when maintaining a long connection.

"Sticky packet" can occur at the sending end or at the receiving end:

Sticky packets at the sender caused by Nagle algorithm: Nagle algorithm is an algorithm to improve network transmission efficiency. Simply put, when we submit a piece of data to TCP for transmission, TCP does not immediately send this piece of data, but waits for a short period of time to see if there is still data to be sent during the waiting period. If so, it will send these two pieces of data at one time.
Packet sticking at the receiving end is caused by untimely or incomplete reception at the receiving end: TCP will store the received data in its own buffer, and then notify the application layer to fetch the data. When the application layer can't take out the TCP data in time for some reasons, several segments of data will be stored in the TCP buffer. When receiving, it will be confiscated clean and complete, resulting in the first data received next time.

(2) Solution

The key to "sticking packets" is that the receiver is uncertain about the size of the packets to be transmitted, so we can packet and unpack the packets.

Packet: packet is to add a packet header to a piece of data, so that the packet is divided into two parts: packet header and packet body (when filtering illegal packets, the packet will add "packet tail" content). The length of the packet header is fixed, and it stores the length of the packet body. According to the fixed length of the packet header and the variable containing the length of the packet body in the packet header, a complete packet can be correctly split.

We can define a protocol ourselves. For example, the first four bytes of a packet are the packet header, which stores the length of the transmitted data.

// socket_stick/proto/proto.go
package proto

import (
	"bufio"
	"bytes"
	"encoding/binary"
)

// Encode encodes the message and encodes the message when sending
func Encode(message string) ([]byte, error) {
	// Read the length of the message and convert it to int32 type (4 bytes)
	var length = int32(len(message))
	var pkg = new(bytes.Buffer)
	// Write header
	err := binary.Write(pkg, binary.LittleEndian, length)
	if err != nil {
		return nil, err
	}
	// Write message entity
	err = binary.Write(pkg, binary.LittleEndian, []byte(message))
	if err != nil {
		return nil, err
	}
	return pkg.Bytes(), nil
}

// Decode decodes the message and decodes the message when received
func Decode(reader *bufio.Reader) (string, error) {
	// Read the length of the message
	lengthByte, _ := reader.Peek(4) // Read the first 4 bytes of data
	lengthBuff := bytes.NewBuffer(lengthByte)
	var length int32
	err := binary.Read(lengthBuff, binary.LittleEndian, &length)
	if err != nil {
		return "", err
	}
	// Buffered returns the number of bytes that are currently readable in the buffer.
	if int32(reader.Buffered()) < length+4 {
		return "", err
	}

	// Read real message data
	pack := make([]byte, int(4+length))
	_, err = reader.Read(pack)
	if err != nil {
		return "", err
	}
	return string(pack[4:]), nil
}

Next, the server and client use the Decode and Encode functions of the proto package defined above to process data respectively.

The server code is as follows:

// socket_stick/server2/main.go

func process(conn net.Conn) {
	defer conn.Close()
	reader := bufio.NewReader(conn)
	for {
		msg, err := proto.Decode(reader)
		if err == io.EOF {
			return
		}
		if err != nil {
			fmt.Println("decode msg failed, err:", err)
			return
		}
		fmt.Println("received client Data sent:", msg)
	}
}

func main() {

	listen, err := net.Listen("tcp", "127.0.0.1:30000")
	if err != nil {
		fmt.Println("listen failed, err:", err)
		return
	}
	defer listen.Close()
	for {
		conn, err := listen.Accept()
		if err != nil {
			fmt.Println("accept failed, err:", err)
			continue
		}
		go process(conn)
	}
}

The client code is as follows:

// socket_stick/client2/main.go

func main() {
	conn, err := net.Dial("tcp", "127.0.0.1:30000")
	if err != nil {
		fmt.Println("dial failed, err", err)
		return
	}
	defer conn.Close()
	for i := 0; i < 20; i++ {
		msg := `Hello, Hello. How are you?`
		data, err := proto.Encode(msg)
		if err != nil {
			fmt.Println("encode msg failed, err:", err)
			return
		}
		conn.Write(data)
	}
}

2, Realize UDP communication

1. UDP protocol

The Chinese name of UDP protocol is User Datagram Protocol, which is OSI (Open System Interconnection) a connectionless transport layer protocol in the reference model, which can directly send and receive data without establishing a connection. It belongs to unreliable and non sequential communication, but UDP protocol has good real-time performance and is usually used in the field of live video broadcasting.

2. UDP server

// UDP/server/main.go

// UDP server side
func main() {
	listen, err := net.ListenUDP("udp", &net.UDPAddr{
		IP:   net.IPv4(0, 0, 0, 0),
		Port: 30000,
	})
	if err != nil {
		fmt.Println("listen failed, err:", err)
		return
	}
	defer listen.Close()
	for {
		var data [1024]byte
		n, addr, err := listen.ReadFromUDP(data[:]) // receive data 
		if err != nil {
			fmt.Println("read udp failed, err:", err)
			continue
		}
		fmt.Printf("data:%v addr:%v count:%v\n", string(data[:n]), addr, n)
		_, err = listen.WriteToUDP(data[:n], addr) // send data
		if err != nil {
			fmt.Println("write to udp failed, err:", err)
			continue
		}
	}
}

3. UDP client

// UDP client
func main() {
	socket, err := net.DialUDP("udp", nil, &net.UDPAddr{
		IP:   net.IPv4(0, 0, 0, 0),
		Port: 30000,
	})
	if err != nil {
		fmt.Println("Failed to connect to the server, err:", err)
		return
	}
	defer socket.Close()
	sendData := []byte("Hello server")
	_, err = socket.Write(sendData) // send data
	if err != nil {
		fmt.Println("Failed to send data, err:", err)
		return
	}
	data := make([]byte, 4096)
	n, remoteAddr, err := socket.ReadFromUDP(data) // receive data 
	if err != nil {
		fmt.Println("Failed to receive data, err:", err)
		return
	}
	fmt.Printf("recv:%v addr:%v count:%v\n", string(data[:n]), remoteAddr, n)
}