Using select and poll/epoll of IO multiplexing to talk about network chat room

Posted by kaszu on Thu, 16 Dec 2021 22:18:45 +0100

IO multiplexing:

The IO multiplexing model is based on the demultiplexing function select provided by the kernel. Using the select function can avoid the problem of polling and waiting in the synchronous non blocking IO model.

select

This function is used to monitor the change of file descriptor - read / write or exception


Parameters:

nfds: Usually set to select The maximum of all file descriptors listened on+1

		Specifies the total number of file descriptors that are listened on

		File descriptor starts at 0

fd_set: File descriptor collection

		fd_set The structure contains only one integer array, each bit of each element of the array(bit)

		Mark a file descriptor

		fd_set The number of file descriptors that can be accommodated is determined by FD_SETSIZE appoint

		Limited select Number of file descriptors processed 

		FD_ZERO(fd_set *fdset); //Clear all bits

		FD_SET(int fd,fd_set *fdset);   //Set fd

		FD_CLR(int fd,fd_set *fdset);   //Clear fd

		int FD_ISSET(int fd,fd_set *fdset); //test

	readfds,writefds,exceptfds Used to record whether to listen or not

		File descriptors for readable, writable, exception events match

	select When the function call returns, the kernel will modify readfds,writefds,excepts

	A collection of file descriptors that retain data readable, writable, and abnormal file descriptors

timeout: 

	set up select Timeout for function

	return select The time remaining after the call returns. If the call fails timeout uncertain

	If timeout If all variable members are 0, then select Return now

	If timeout Value is NULL,select Blocks until a file descriptor is ready

	struct timeval{

		long tv_sec;   //Seconds

		long tv_usec;  //Microseconds

	}

Return value:

	select Return ready on success(Readable, writable, and exception)Sum of file descriptors

	Returns 0 if any file descriptors are ready within the timeout

	select Failure Return-1 And set errno

	stay select While waiting, the program receives a signal, select Return now-1,

	errno by EINTR

principle:

	Give the set of file descriptors to be monitored to the kernel for testing, and keep the ready file descriptors



	Each time, you need to add all the file descriptors you need to listen to to to the file descriptor set again

	The kernel needs to traverse all file descriptors

	When select After returning, you need to cycle through all the file descriptors to determine whether they are ready

	

	When the number of file descriptors increases, the efficiency actually decreases sharply

	select Also received FD_SETSIZE Limitations of 

	When select After returning, there is no parallel, but serial, one by one to receive and forward the data of the client

Relationship between select() function and Linux Driver

When the user calls the select system call, the select system call will call poll first_ Initwait (&table), then call the struct file_ in the driver. FOP - > poll function under operations. Poll should be called in this function_ Wait(), add the current to a waiting queue (call poll_wait() here), and check whether it is valid. If it is invalid, call schedule_timeout(); Go to sleep. After the event occurs, schedule_ When timeout () comes back, call FOP - > poll (). When it is checked that it can run, call poll_ freewait(&table); This completes the select system call. It is important to check whether FOP - > poll () is ready. If so, return the corresponding flag.

chat room

#include <stdio.h>
#include <stdlib.h>
#include <stdbool.h>
#include <string.h>
#include <errno.h>
#include <assert.h>
#include <unistd.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <sys/select.h>

#define NAME_LEN 48
#define MAX_CLIENTS 100
#define MSG_LEN 1024
typedef struct Client{
	int fd;
	struct sockaddr_in addr;
	char name[NAME_LEN];
}Client;

//Global variables are accessed by every thread 
Client gcls[MAX_CLIENTS] = {};
int size = 0;

int init_server(const char *ip,unsigned short int port){
	int fd = socket(AF_INET,SOCK_STREAM,0);
	assert(fd != -1);
	struct sockaddr_in addr = {};
	addr.sin_family = AF_INET;
	addr.sin_port = htons(port);
	addr.sin_addr.s_addr = inet_addr(ip);
	socklen_t len = sizeof(addr);
	int ret = bind(fd,(const struct sockaddr*)&addr,len);
	assert(ret != -1);
	ret = listen(fd,MAX_CLIENTS);
	assert(ret != -1);
	return fd;
}

void broadcast(int fd,const char *msg){
	int i;
	for(i=0;i<size;i++){
		if(fd != gcls[i].fd){
			send(gcls[i].fd,msg,strlen(msg)+1,0);	
		}	
	}
}

void remove_client(int fd){
	int i;
	for(i=0;i<size;i++){
		if(fd == gcls[i].fd){
			close(fd);
			gcls[i] = gcls[--size];
			break;
		}	
	}
}

void select_fd(int fd){
	Client cls = {};
	socklen_t len = sizeof(cls.addr);
	int ret = 0,i,cnt;
	fd_set readfds; //Set of readable file descriptors
	int maxfd = 0;  //Record the largest file descriptor
	while(true){
		FD_ZERO(&readfds);//File descriptor collection empty
		maxfd = fd;
		FD_SET(fd,&readfds);//The fd of the server is used to determine whether there is a client connection
		for(i=0;i<size;i++){
			FD_SET(gcls[i].fd,&readfds);//File descriptor that interacts with the client
			if(gcls[i].fd > maxfd)
				maxfd = gcls[i].fd;
		}
		cnt = select(maxfd+1,&readfds,NULL,NULL,NULL);//block
		if(FD_ISSET(fd,&readfds)){//The fd client of the server is connected
			cls.fd = accept(fd,(struct sockaddr*)&cls.addr,&len);
			if(cls.fd != -1){
				gcls[size++] = cls;	//I didn't send my network name when I entered the chat room
			}
			--cnt;
		}
		for(i=0;i<size&&cnt>0;i++){
			if(FD_ISSET(gcls[i].fd,&readfds)){//Ensure that there is a data readable thread pool
				--cnt;
				char msg[MSG_LEN] = {};
				if(strcmp(gcls[i].name,"")==0){//Send network name
					ret = recv(gcls[i].fd,msg,MSG_LEN,0);
					if(ret <= 0){
						remove_client(gcls[i].fd);
						continue;
					}
					strcpy(gcls[i].name,msg);
					strcat(msg," Enter the chat room,Welcome!");
				}else{//send data
					strcpy(msg,gcls[i].name);
					strcat(msg,":");
					int len = strlen(msg);
					ret = recv(gcls[i].fd,msg+len,MSG_LEN-len,0);
					if(ret <= 0){
						msg[len-1] = '\0';
						strcat(msg," Quit group chat,Welcome everyone!");
					}
				}
				broadcast(gcls[i].fd,msg);
				if(ret <= 0){
					remove_client(gcls[i].fd);	
				}
			}	
		}
	}
}

int main(int argc,char *argv[]){
	if(argc < 3){
		printf("%s ip port\n",argv[0]);
		return -1;
	}
	int fd = init_server(argv[1],atoi(argv[2]));
	select_fd(fd);
	return 0;	
}


poll/epoll

Poll and select have similar functions, but poll is efficient. Poll does not need to add all file descriptors to the collection (array) every time
Poll notifies the user in different ways. Select deletes the file descriptor that is not ready from the collection. The poll kernel directly sets the events attribute of each data in the collection. The same thing is that after calling the poll/select function, you need to traverse all the file descriptors to determine whether it is ready. Poll() accepts a pointer to the structure 'struct pollfd' list, This includes the file descriptors and events you want to test. Events are determined by a bitmask in the event domain of the structure. The current structure will be filled in after the call and returned after the event. The "poll.h" file in SVR4 (possibly earlier versions) contains some macro definitions for determining events. The waiting time of the event is accurate to milliseconds (but the confusing thing is that the type of waiting time is int). When the waiting time is 0, the poll() function returns immediately, and - 1 causes poll() to hang until a specified event occurs. The following is the structure of pollfd.

Parameters:

	fds: It's a struct pollfd Array of structure types

		struct pollfd{

			int fd;            //File descriptor

			short events;      //Registered events are readable and writable

			short revents;     //The events that actually occur are populated by the kernel

		};

		POLLIN         Data readability

		POLLOUT        Data writable

	nfds:

		Array length

	timeout:

		Number of milliseconds to wait for timeout

		-1   block

		0    Return now

Return value:

	and select Number of file descriptors with the same ready state

epoll yes Linux Peculiar I/O Multiplexing function

epoll Register the events on the file descriptor concerned by the user into the event table of the kernel

You don't need to be like select/poll The file descriptor set and event set are retransmitted each time

epoll There is no need to traverse all the file descriptor sets after the call is completed

epoll An additional file descriptor is required to uniquely identify the event table in the kernel

pool to realize network chat room

#include <stdio.h>
#include <stdlib.h>
#include <stdbool.h>
#include <string.h>
#include <errno.h>
#include <assert.h>
#include <unistd.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <poll.h>

#define NAME_LEN 48
#define MAX_CLIENTS 100
#define MSG_LEN 1024
typedef struct Client{
	int fd;
	struct sockaddr_in addr;
	char name[NAME_LEN];
}Client;

Client gcls[MAX_CLIENTS+1] = {};//gcls[0] does not store values
struct pollfd fds[MAX_CLIENTS+1] = {};
int size = 0;

int init_server(const char *ip,unsigned short int port){
	int fd = socket(AF_INET,SOCK_STREAM,0);
	assert(fd != -1);
	struct sockaddr_in addr = {};
	addr.sin_family = AF_INET;
	addr.sin_port = htons(port);
	addr.sin_addr.s_addr = inet_addr(ip);
	socklen_t len = sizeof(addr);
	int ret = bind(fd,(const struct sockaddr*)&addr,len);
	assert(ret != -1);
	ret = listen(fd,MAX_CLIENTS);
	assert(ret != -1);
	return fd;
}

void broadcast(int fd,const char *msg){
	int i;
	for(i=1;i<size;i++){
		if(fd != gcls[i].fd){
			send(gcls[i].fd,msg,strlen(msg)+1,0);	
		}	
	}
}

void accept_client(int fd){
	struct sockaddr_in addr = {};
	socklen_t len = sizeof(addr);
	int cfd = accept(fd,(struct sockaddr*)&addr,&len);
	if(cfd != -1){
		Client cls = {};
		cls.fd = cfd;
		cls.addr = addr;
		gcls[size] = cls;
		fds[size].fd = cfd;
		fds[size].events = POLLIN;
		++size;
	}
}

int recv_data(int index){
	int fd = fds[index].fd;	
	char msg[MSG_LEN] = {};
	int ret = 0;
	if(strcmp(gcls[index].name,"")==0){
		ret = recv(fd,msg,MSG_LEN,0);
		if(ret <= 0){
			return 0;
		}
		strcpy(gcls[index].name,msg);
		strcat(msg," Enter the chat room,Welcome!");
	}else{
		strcpy(msg,gcls[index].name);
		strcat(msg,":");
		int len = strlen(msg);
		ret = recv(fd,msg+len,MSG_LEN-len,0);
		if(ret <= 0){
			msg[--len] = '\0';
			strcat(msg," Exit chat,Welcome everyone!");	
		}
	}
	broadcast(fd,msg);
	if(ret <= 0)
		return 0;
	return 1;
}
void select_fd(int fd){
	int ret = 0,i,cnt;
	//fds  struct pollfd fds[]  
	fds[0].fd = fd;        //File descriptor
	fds[0].events = POLLIN;//Listening events
	size = 1;
	while(true){
		cnt = poll(fds,size,-1);//
		for(i=0;i<size&&cnt>0;i++){
			if(fds[i].revents & POLLIN){//fds[i].fd ready
				--cnt;
				if(fds[i].fd == fd){//Connect to the client
					accept_client(fd);//Receive client connection request				
				}else{//data
					if(recv_data(i)==0){
						gcls[i] = gcls[size-1];
						fds[i] = fds[size-1];
						--size;
						--i;
					}
				}
			}	
		}
	}
}

int main(int argc,char *argv[]){
	if(argc < 3){
		printf("%s ip port\n",argv[0]);
		return -1;
	}
	int fd = init_server(argv[1],atoi(argv[2]));
	select_fd(fd);
	return 0;	
}


epoll to realize network chat room (QQ group)

#include <stdio.h>
#include <stdlib.h>
#include <stdbool.h>
#include <string.h>
#include <errno.h>
#include <assert.h>
#include <unistd.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <sys/epoll.h>

#define NAME_LEN 48
#define MAX_CLIENTS 100
#define MSG_LEN 1024
typedef struct Client{
	int fd;
	struct sockaddr_in addr;
	char name[NAME_LEN];
}Client;

Client gcls[MAX_CLIENTS+1] = {};
int size = 0;

int init_server(const char *ip,unsigned short int port){
	int fd = socket(AF_INET,SOCK_STREAM,0);
	assert(fd != -1);
	struct sockaddr_in addr = {};
	addr.sin_family = AF_INET;
	addr.sin_port = htons(port);
	addr.sin_addr.s_addr = inet_addr(ip);
	socklen_t len = sizeof(addr);
	int ret = bind(fd,(const struct sockaddr*)&addr,len);
	assert(ret != -1);
	ret = listen(fd,MAX_CLIENTS);
	assert(ret != -1);
	return fd;
}

void broadcast(int fd,const char *msg){
	int i;
	for(i=0;i<size;i++){
		if(fd != gcls[i].fd){
			send(gcls[i].fd,msg,strlen(msg)+1,0);	
		}	
	}
}

void accept_client(int fd,int epfd){
	struct sockaddr_in addr = {};
	socklen_t len = sizeof(addr);
	int cfd = accept(fd,(struct sockaddr*)&addr,&len);
	if(cfd != -1){
		Client cls = {};
		cls.fd = cfd;
		cls.addr = addr;
		gcls[size] = cls;
		++size;
		struct epoll_event event = {};
		event.events = EPOLLIN;
		event.data.fd = cfd;
		int ret = epoll_ctl(epfd,EPOLL_CTL_ADD,cfd,&event);
		if(ret == -1){
			perror("epoll_ctl");	
		}
	}
}

int recv_data(int fd){
	int index = 0;
	for(;index<size;index++){
		if(gcls[index].fd == fd){
			break;	
		}	
	}
	char msg[MSG_LEN] = {};
	int ret = 0;
	if(strcmp(gcls[index].name,"")==0){
		ret = recv(fd,msg,MSG_LEN,0);
		if(ret <= 0){
			return 0;
		}
		strcpy(gcls[index].name,msg);
		strcat(msg," Enter the chat room,Welcome!");
	}else{
		strcpy(msg,gcls[index].name);
		strcat(msg,":");
		int len = strlen(msg);
		ret = recv(fd,msg+len,MSG_LEN-len,0);
		if(ret <= 0){
			msg[--len] = '\0';
			strcat(msg," Exit chat,Welcome everyone!");	
		}
	}
	broadcast(fd,msg);
	if(ret <= 0)
		return 0;
	return 1;
}
void select_fd(int fd){
	int epfd = epoll_create(MAX_CLIENTS);
	if(epfd == -1){
		perror("epoll_create");
		return;	
	}
	struct epoll_event event = {};
	event.events = EPOLLIN;  //Read event
	event.data.fd = fd;     //user data
	int ret = epoll_ctl(epfd,EPOLL_CTL_ADD,fd,&event);
	if(ret == -1){
		perror("epoll_ctl");
		return ;
	}
	struct epoll_event events[MAX_CLIENTS+1] = {};
	int i;
	while(true){
		ret = epoll_wait(epfd,events,MAX_CLIENTS+1,-1);
		if(ret == -1){
			perror("epoll_wait");
			break;
		}
		for(i=0;i<ret;i++){
			if(events[i].data.fd == fd){//There is a client connection
				accept_client(fd,epfd);
			}else{//There is data to receive
				if(events[i].events & EPOLLIN){
					ret = recv_data(events[i].data.fd);
					if(ret == 0){
						struct epoll_event ev = {};
						ev.events = EPOLLIN;
						ev.data.fd = events[i].data.fd;
						ret = epoll_ctl(epfd,EPOLL_CTL_DEL,events[i].data.fd,&ev);
						if(ret == -1){
							perror("epoll_ctl");	
						}
					}
				}
			}
		}
	}
}

int main(int argc,char *argv[]){
	if(argc < 3){
		printf("%s ip port\n",argv[0]);
		return -1;
	}
	int fd = init_server(argv[1],atoi(argv[2]));
	select_fd(fd);
	return 0;	
}


summary

select:

	The user passes in readable, writable and abnormal event file descriptor sets through three parameters

	The kernel feeds back the ready events by modifying these parameters online

	Enables the user to call select These three parameters need to be reset

poll:

	All event types are handled uniformly, so only one event set parameter is required.

	You can use the structure events Incoming event 

	The kernel modifies the structure's revents Feedback on events in which

	The user does not need to reset every time events parameter

epoll:

	The kernel directly manages all events registered by users through an event table

	So every call epoll_wait There is no need to repeatedly pass in the event of user registration

	epoll_wait parameter events Only used to feed back ready events

	It is not necessary to traverse all the file descriptor sets

The time complexity of the application index file descriptor

select:  O(n)

poll:    O(n)

epoll:   O(1)

Maximum supported file descriptors

select: Restricted by many parties

poll:   65535

epoll:  65535

Working mode

select: LT

poll:   LT

epoll:  The default is LT  support ET Efficient mode

Kernel Implementation and efficiency

select:

	Polling is used to detect ready events, and the time complexity of the algorithm is O(n)

poll:

	Polling is used to detect ready events, and the time complexity of the algorithm is O(n)

epoll:

	The callback method is used to detect the ready event, and the time complexity of the algorithm is O(1)

Topics: PHP Database Multithreading thread pool epoll