Today is another day submerged by pouring demand.

Does anyone know where to sign up for the chicken soup class of "I cut 18 demands for me in three sentences" and want to apply.

The class of "understanding applause" is even more difficult.

Going further, back to our topic today, let's understand the catalogue of this article.

After the code executes send successfully, will the data be sent out?

Before answering this question, you need to know what a Socket buffer is.

Socket buffer

What is a socket buffer

When programming, if we want to establish a connection with an IP, we need to call the socket API provided by the operating system.

At the operating system level, socket can be understood as a file.

We can do some method operations on this file.

Using the listen method, you can let the program listen to the connections of other clients as a server.

With connect, you can connect to the server as a client.

Send or write can send data, recv or read can receive data.

After the connection is established, the socket file is like the "agent" of the remote machine. For example, if we want to send something to the remote service, we only need to write to the file.

After writing this file, the rest of the sending work is naturally completed by the operating system kernel.

Since it is written to the operating system, the operating system needs to provide a place for users to write. Similarly, receiving messages is the same.

This place is the socket buffer.

When the user sends a message, it is written to the send buffer

When the user receives a message, he writes it to the recv buffer (receive buffer)

In other words, a socket will have two buffers, one for sending and one for receiving. Because this is a first in first out structure, it is sometimes called send and receive queues.

How to observe socket buffer

If you want to view the socket buffer, you can execute the netstat -nt command in the linux environment.

# netstat -nt
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State      
tcp        0     60 172.22.66.69:22         122.14.220.252:59889    ESTABLISHED

The above shows that there is a connection with the protocol (Proto) type of TCP, as well as the IP information of Local Address and Foreign Address. The State is connected.

In addition, Send-Q is the send buffer. The following number 60 means that 60 bytes are not sent in the send buffer at present. Recv-Q represents the receive buffer, which is empty at this time, and the data is received by the application process.

TCP part

[dynamic diagram buffer sending and receiving process, TCP sending and receiving process]

After establishing a connection using TCP, we usually use send to send data.

int main(int argc, char *argv[])
{
    // Create socket
    sockfd=socket(AF_INET,SOCK_STREAM, 0))
     
    // Establish connection  
    connect(sockfd, The server ip information, sizeof(server))  
   
    // Execute send to send a message
    send(sockfd,str,sizeof(str),0))  

    // Close socket
    close(sockfd);
  
    return 0;
}

The above is a piece of pseudo code, which is only used to show the general logic. After establishing the connection, we will generally execute the send method in the code. Will the message be sent to the opposite machine immediately?

Will the bytes sent by send be sent immediately?

The answer is not sure! After send ing, the data is only copied to the socket buffer. When and how much data will be sent depends on the arrangement of the operating system.

In the user process, the program will enter the kernel state from the user state by operating the socket, and the send method will send the data all the way to the transport layer. After recognizing that it is a TCP protocol, TCP is called_ Sendmsg method.

// net/ipv4/tcp.c
// A lot of logic is omitted below
int tcp_sendmsg()
{  
  // If there is room for data
  if (skb_availroom(skb) > 0) {
    // Try to copy the data to be sent to the send buffer
    err = skb_add_data_nocache(sk, skb, from, copy);
  }  
  // The following is the logic code you are trying to send. Omit it first	 
}

In TCP_ In sendmsg, the core work is to put the data to be sent into the sending buffer in order, and then judge whether to send data according to the actual situation (such as congestion window). If you do not send data, return directly at this time.

What happens if the buffer is full

In the case mentioned above, there is enough space in the send buffer to copy the data to be sent.

What happens if the send buffer space is insufficient or full and the send is performed?

There are two cases.

First, when creating a socket, you can set whether it is blocking or non blocking.

int s = socket(AF_INET, SOCK_STREAM | SOCK_NONBLOCK, IPPROTO_TCP);

For example, through the above code, you can set the socket to non blocking (SOCK_NONBLOCK).

When the send buffer is full, if send is also executed to the socket

• If the socket is blocked at this time, the program will wait and die there until a new cache space is released, continue to copy the data in, and then return.

• If the socket is non blocking at this time, the program will immediately return an EAGAIN error message, which means try again. Now the buffer is full, don't wait. Try again later.

We can simply look at how the source code is implemented. Or go back to just now_ Sendmsg sending method.

int tcp_sendmsg()
{  
  if (skb_availroom(skb) > 0) {
    // .. Execute balabla if there are enough buffers
  } else {
    // If there is no space in the transmit buffer, wait until there is space. As for the wait mode, it is divided into blocking and non blocking
    if ((err = sk_stream_wait_memory(sk, &timeo)) != 0)
        goto do_error;
  }   
}		

Sk mentioned in it_ stream_ wait_ Memory will decide whether the socket is blocked or not. Wait a while and then return.

int sk_stream_wait_memory(struct sock *sk, long *timeo_p)
{
	while (1) {
    // In non blocking mode, wait until the timeout returns to EAGAIN
		if (Wait timeout))
			return -EAGAIN;     
     // When blocking waiting, it will not jump out until there is enough space in the send buffer
		if (sk_stream_memory_free(sk) && !vm_wait)
			break;
	}
	return err;
}

What happens if the receive buffer is empty?

The same is true for the receive buffer.

When the receive buffer is empty, if recv is also executed to the socket

• If the socket is blocked at this time, the program will wait there until there is data in the receive buffer, copy the data from the receive buffer to the user buffer, and then return.

• If the socket is non blocking at this time, the program will immediately return an EAGAIN error message.

Here's a chart to summarize, so that you can save it for the interview. Hahaha.

What happens if there is still data in the socket buffer and the close is executed?

First of all, we should know that under normal circumstances, both the send buffer and the receive buffer should be empty.

If the transmit and receive buffers are not empty for a long time, it indicates that there is data accumulation. This is often due to some network problems or user application layer problems, resulting in abnormal data processing.

Under normal circumstances, if the socket buffer is empty, execute close. It triggers four waves.

This is also the content of the old eight part essay of the interview. Here we only need to pay attention to the first wave and send FIN.

What happens if close is executed when there is data in the receive buffer?

When the socket is closed, the main logic is TCP_ Implemented in close().

Let's start with the conclusion that there are two main situations in the closing process:

• If there is still unread data in the receive buffer, the data in the receive buffer will be cleared first, and then an RST will be sent to the opposite end.
• If the receive buffer is empty, TCP is called_ send_ Fin () starts the first wave of the four wave process.

void tcp_close(struct sock *sk, long timeout)
{
  // If there is data in the receive buffer, empty the data
	while ((skb = __skb_dequeue(&sk->sk_receive_queue)) != NULL) {
		u32 len = TCP_SKB_CB(skb)->end_seq - TCP_SKB_CB(skb)->seq -
			  tcp_hdr(skb)->fin;
		data_was_unread += len;
		__kfree_skb(skb);
	}

   if (data_was_unread) {
    // If the data in the receive buffer is cleared, send RST
		tcp_send_active_reset(sk, sk->sk_allocation);
	 } else if (tcp_close_state(sk)) {
    // Wave four times normally and send FIN
		tcp_send_fin(sk);
	}
	// Waiting to close
	sk_stream_wait_close(sk, timeout);
}

What happens if close is executed when there is data in the send buffer?

Previously, I thought that in this case, the kernel would empty the send buffer data and wave four times.

But this is not the case with the source code.

void tcp_send_fin(struct sock *sk)
{
  // Get the last piece of data in the send buffer
	struct sk_buff *skb, *tskb = tcp_write_queue_tail(sk);
	struct tcp_sock *tp = tcp_sk(sk);

  // If there is still data in the send buffer
	if (tskb && (tcp_send_head(sk) || sk_under_memory_pressure(sk))) {
		TCP_SKB_CB(tskb)->tcp_flags |= TCPHDR_FIN; // Set the last data value to FIN 
		TCP_SKB_CB(tskb)->end_seq++;
		tp->write_seq++;
	}  else {
    // If there is no data in the send buffer, create a FIN packet
  }
  // send data
	__tcp_push_pending_frames(sk, tcp_current_mss(sk), TCP_NAGLE_OFF);
}

At this time, some data is not sent out, and the kernel will take out the last data block in the send buffer. Then set to FIN.

The socket buffer is a first in first out queue, which means that the kernel will wait for the TCP layer to send all the data in the sending buffer quietly, and finally execute the first wave (FIN packet) of four waves.

It should be noted that only when the receive buffer is empty can we get to tcp_send_fin() . Only after entering this method can we consider whether the send buffer is empty.

UDP part

Does UDP also have a buffer

With TCP finished, let's talk about UDP. This pair of good friends are also important protocols in the transport layer. Since TCP has send and receive buffers mentioned earlier, does UDP have?

I used to think.

"Each UDP socket has a receive buffer, but there is no send buffer. Conceptually, it is sent as long as there is data, regardless of whether the other party can receive it correctly, so there is no buffer and there is no need to send a buffer."

Then I found out I was wrong.

UDP socket is also a socket. A socket has two buffers: receive and send. It doesn't matter what agreement you use.

Whether it is used or not is one thing.

UDP no send buffer?

In fact, UDP not only has a send buffer, but also uses a send buffer.

Under normal circumstances, the data will be directly copied to the transmission buffer and sent directly.

Another case is to set an MSG when sending data_ Mark of more.

ssize_t send(int sock, const void *buf, size_t len, int flags); // Set flag to MSG_MORE

The general meaning is to tell the kernel that there are more messages to be sent together later. Don't worry about sending them first. At this time, the kernel will cache this data with the send buffer first, and then send it together when the application layer says ok later.

We can look at the source code.

int udp_sendmsg()
{
	// If corkreq is true, it means MSG_MORE mode, which only organizes messages and does not send them;
	int corkreq = up->corkflag || msg->msg_flags&MSG_MORE；
    
	//  The data to be sent is divided according to the size of MTU, with one skb for each segment; And these
	//  skb will be put into the sending buffer of the socket; This function only organizes packets and does not send them.
	err = ip_append_data(sk, fl4, getfrag, msg->msg_iov, ulen,
			     sizeof(struct udphdr), &ipc, &rt,
			     corkreq ? msg->msg_flags|MSG_MORE : msg->msg_flags);

	// MSG is not enabled_ More feature, the data in the send queue is directly sent to the IP. 
    if (!corkreq)
		err = udp_push_pending_frames(sk);

}

So, whether it's MSG or not_ Both more and IP will put the data into the sending queue first, and then consider whether to send it immediately according to the actual situation.

In most cases, we don't use MSG_MORE, that is, send a packet directly when a packet comes. In terms of this behavior, although UDP uses the send buffer, it does not actually play the role of "buffer".

last

I've only written this article for 20 hours. Painting is just painting vomiting. I get up at 7 o'clock every morning and write for more than an hour before going to work.

Brothers are from their own families. They like it or not. It doesn't matter whether they are watching it or not. Just be happy.

I'm watching and praising something. I don't care very much. Really, really, don't believe it.

No, it really doesn't matter.

Brothers, don't care.

I'm a little white with different opinions. I'll see you next time!

Recently, I want to build a group. Considering that many technical groups have been added, we decided to build a chat, fishing and water bucket map group.

Although my technology can't compare with the big guys, I have a lot of expression bags!

Everyone has a lunch break or gets off work.

You can brush the group occasionally, chat with each other and brag, which is very decompression.

Make more friends in the group. The interviewer for your next interview is likely to be in the group.

Add me and pull you into the group.

Stop talking, let's choke in the ocean of knowledge

Attention to the official account: debug