1. Preface
Review of previous blogs:
C/C++ Server/Background Development Learning Route Summary and Preparation
What is a server? Server Classification and Building a Simple Server System
[C/C++ Server Development] socket Network Programming Function Interface Details
[C/C++ Server Development] Flexible application of socket network programming function interface
Previous servers only provided simple functionality, and we can enrich their functionality to provide richer functionality. For example, a web server can provide web page data, a file server can provide file downloads, and so on.
In addition, previous servers could only respond to requests from one client at a time, so we need to consider using multiple processes or threads to improve the server model so that it can receive requests from multiple clients at the same time.
2. Servers with richer functions
The function of the server needs to be determined according to our needs. Here we are just providing a way of thinking, a direction.
For example, we save a copy of data on the database side, and clients can query the data by connecting to the server side.
server.cpp
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <unistd.h>
#include <arpa/inet.h>
#include <sys/socket.h>
#include <netinet/in.h>
int main(){
//Create Socket
int serv_sock = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
//Bind sockets to IP and ports
struct sockaddr_in serv_addr;
memset(&serv_addr, 0, sizeof(serv_addr)); //Each byte is filled with 0
serv_addr.sin_family = AF_INET; //Use IPv4 address
serv_addr.sin_addr.s_addr = inet_addr("127.0.0.1"); //Specific IP address
serv_addr.sin_port = htons(1234); //port
bind(serv_sock, (struct sockaddr*)&serv_addr, sizeof(serv_addr));
//Enter listening state, wait for user to initiate request
listen(serv_sock, 20);
char bufferrev[100];
memset(bufferrev, 0x00, sizeof(bufferrev));
char* buffersnd;
while(1) {
//Receive client requests
struct sockaddr_in clnt_addr;
socklen_t clnt_addr_size = sizeof(clnt_addr);
int clnt_sock = accept(serv_sock, (struct sockaddr*)&clnt_addr, &clnt_addr_size);
int len = read(clnt_sock, bufferrev, 100-1);
printf("receive data: %s\n", bufferrev);
if (strcmp(bufferrev, "wuhan") == 0) {
strcpy(buffersnd, "you want know wuhan!");
} else if (strcmp(bufferrev, "guangzhou") == 0) {
strcpy(buffersnd, "you want know guangzhou!");
} else if (strcmp(bufferrev, "shenzhen") == 0) {
strcpy(buffersnd, "you want know shenzhen!");
} else if (strcmp(bufferrev, "shagnhai") == 0) {
strcpy(buffersnd, "you want know shanghai!");
} else {strcpy(buffersnd, "no data about that!");}
//Send data back to client
printf("server send data to client!\n");
write(clnt_sock, buffersnd, sizeof(buffersnd));
//Reset Cache
memset(bufferrev, 0x00, sizeof(bufferrev));
memset(buffersnd, 0x00, sizeof(buffersnd));
//Close Client Socket
close(clnt_sock);
}
//Close server socket
close(serv_sock);
return 0;
}
client.cpp
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <unistd.h>
#include <arpa/inet.h>
#include <sys/socket.h>
int main(){
//Initiate a request to a server (specific IP and port)
struct sockaddr_in serv_addr;
memset(&serv_addr, 0, sizeof(serv_addr)); //Each byte is filled with 0
serv_addr.sin_family = AF_INET; //Use IPv4 address
serv_addr.sin_addr.s_addr = inet_addr("127.0.0.1"); //Specific IP address
serv_addr.sin_port = htons(1234); //port
char bufSend[100] = {0};
char bufRecv[100] = {0};
while(1) {
//Create Socket
int sock = socket(AF_INET, SOCK_STREAM, 0);
connect(sock, (struct sockaddr*)&serv_addr, sizeof(serv_addr));
//Gets a string entered by the user and sends it to the server
printf("please choose a city you want know!\n");
printf("wuhan guangzhou shanghai shenzhen\n");
printf("Input a city name: ");
scanf("%s", bufSend);
//gets(bufSend);
write(sock, bufSend, sizeof(bufSend));
//Read data returned by the server
read(sock, bufRecv, sizeof(bufRecv)-1);
printf("Message form server: %s\n", bufRecv);
//Close
close(sock);
}
return 0;
}
3. Enable the server to respond to multiple client requests simultaneously
1. Single Thread/Process
In the process of TCP communication, the server-side can establish connections with multiple clients and communicate with each other at the same time after starting up, but when introducing the TCP communication process, the server code provided cannot meet this requirement. First, simply look at the processing ideas of the server code before, and then analyze the drawbacks in the code:
// server.c
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <arpa/inet.h>
int main()
{
// 1. Create a listening socket
int lfd = socket(AF_INET, SOCK_STREAM, 0);
// 2. Bind the socket() return value to the local IP port
struct sockaddr_in addr;
addr.sin_family = AF_INET;
addr.sin_port = htons(10000); // Large End Port
// INADDR_ANY represents all IP addresses of the local computer, assuming that three network cards have three IP addresses
// This macro can represent any IP address
addr.sin_addr.s_addr = INADDR_ANY; // The value of this macro is 0 == 0.0.0.0
int ret = bind(lfd, (struct sockaddr*)&addr, sizeof(addr));
// 3. Set up monitoring
ret = listen(lfd, 128);
// 4. Block waiting and accept client connections
struct sockaddr_in cliaddr;
int clilen = sizeof(cliaddr);
int cfd = accept(lfd, (struct sockaddr*)&cliaddr, &clilen);
// 5. Communicate with clients
while(1)
{
// receive data
char buf[1024];
memset(buf, 0, sizeof(buf));
int len = read(cfd, buf, sizeof(buf));
if(len > 0)
{
printf("Client say: %s\n", buf);
write(cfd, buf, len);
}
else if(len == 0)
{
printf("Client disconnected...\n");
break;
}
else
{
perror("read");
break;
}
}
close(cfd);
close(lfd);
return 0;
}
There are three functions used in the code above that can cause a program to block:
- accept(): Blocks the current process/thread if there is no new client connection on the server side, and establishes the connection if a new connection is detected to unblock
- read(): If there is no data in the read buffer corresponding to the socket for communication, the current process/thread is blocked, data is detected to be unblocked and data is received
- write(): If the socket write buffer for communication is full, block the current process/thread (which is less common)
If a connection needs to be established with a client that initiates a new connection request, the accept() function must be called in a loop on the server side. In addition, clients that have already established a connection with the server need to communicate with the server. The blockage when sending data can be ignored, and the program can be blocked when data is not received. This is a very contradictory situation and can be accepted() Blocked cannot communicate, and blocked by read() cannot establish a new connection with the client. Therefore, it is concluded that in a single-threaded/single-process scenario, the server cannot handle multiple connections, and there are many solutions, four of which are commonly used:
- Using multithreaded implementation
- Using multiprocess implementations
- Implement using IO multiplexing
- Using IO Multiplex Transfer+Multithread Implementation
2. Multi-process concurrency
If you want to write a concurrent server program with a multi-process version, first of all, consider what roles the multiple processes created will play so that you can log in to the program. There are two roles on the Tcp server side: listening and communication, listening is a continuous action, establishing a connection if there is a new connection, and blocking if there is no new connection. With respect to communication, multiple clients are required at the same time, so multiple processes are needed to achieve a mutually exclusive effect. There are also two categories of processes: parent and child processes, through which we can assign processes:
Parent process:
- Responsible for listening and handling client connection requests, i.e. looping the accept() function in the parent process
- Create a subprocess: Create a new connection and create a new subprocess to communicate with the corresponding client
- Recycle child process resources: The child process exits to reclaim its kernel PCB resources to prevent zombie processes
Subprocess:
- Responsible for communication, file descriptors obtained after a new connection is established based on the parent process, and the corresponding client completes the receiving and sending of data.
- Send data: send() / write()
- Receive data: recv() / read()
In a multi-process version of a server-side program, multiple processes are related. For related processes, you also need to understand what resources they have to inherit, which resources are exclusive, and some other details:
- The child process is a copy of the parent process, and in the kernel PCB of the child process, the file descriptors can also be copied, so there is also a file descriptor that the parent process can use in the child process, and they can be used to do the same thing as the parent process.
- Parent-child processes use separate virtual address spaces, so all resources are exclusive
- To save system resources, resources that can only be used by the parent process can be freed in the child process, as can the parent process.
- Since acceptance () needs to be done in the parent process and child process resources are released, signal processing can be used if you want to be more efficient
The example code for a multiprocess version of a concurrent TCP server is as follows:
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <arpa/inet.h>
#include <signal.h>
#include <sys/wait.h>
#include <errno.h>
// Signal processing function
void callback(int num)
{
while(1)
{
pid_t pid = waitpid(-1, NULL, WNOHANG);
if(pid <= 0)
{
printf("Subprocess is running, Or the child process is recycled\n");
break;
}
printf("child die, pid = %d\n", pid);
}
}
int childWork(int cfd);
int main()
{
// 1. Create a listening socket
int lfd = socket(AF_INET, SOCK_STREAM, 0);
if(lfd == -1)
{
perror("socket");
exit(0);
}
// 2. Bind the socket() return value to the local IP port
struct sockaddr_in addr;
addr.sin_family = AF_INET;
addr.sin_port = htons(10000); // Large End Port
// INADDR_ANY represents all IP addresses of the local computer, assuming that three network cards have three IP addresses
// This macro can represent any IP address
// This macro is typically used for local binding operations
addr.sin_addr.s_addr = INADDR_ANY; // The value of this macro is 0 == 0.0.0.0
// inet_pton(AF_INET, "192.168.237.131", &addr.sin_addr.s_addr);
int ret = bind(lfd, (struct sockaddr*)&addr, sizeof(addr));
if(ret == -1)
{
perror("bind");
exit(0);
}
// 3. Set up monitoring
ret = listen(lfd, 128);
if(ret == -1)
{
perror("listen");
exit(0);
}
// Capture of registration signal
struct sigaction act;
act.sa_flags = 0;
act.sa_handler = callback;
sigemptyset(&act.sa_mask);
sigaction(SIGCHLD, &act, NULL);
// Accept multiple client connections, call accept iteratively if necessary
while(1)
{
// 4. Block waiting and accept client connections
struct sockaddr_in cliaddr;
int clilen = sizeof(cliaddr);
int cfd = accept(lfd, (struct sockaddr*)&cliaddr, &clilen);
if(cfd == -1)
{
if(errno == EINTR)
{
// The accept call was signaled off, unblocked, and returned -1
// Re-invoke accept once
continue;
}
perror("accept");
exit(0);
}
// Print client address information
char ip[24] = {0};
printf("Client's IP address: %s, port: %d\n",
inet_ntop(AF_INET, &cliaddr.sin_addr.s_addr, ip, sizeof(ip)),
ntohs(cliaddr.sin_port));
// A new connection has been established, creating a subprocess to communicate with this client
pid_t pid = fork();
if(pid == 0)
{
// Subprocess->Communicate with Client
// The file descriptor cfd of the communication is copied into the child process
// Subprocess is not responsible for listening
close(lfd);
while(1)
{
int ret = childWork(cfd);
if(ret <=0)
{
break;
}
}
// Exit subprocess
close(cfd);
exit(0);
}
else if(pid > 0)
{
// Parent process does not communicate with client
close(cfd);
}
}
return 0;
}
// 5. Communicate with clients
int childWork(int cfd)
{
// receive data
char buf[1024];
memset(buf, 0, sizeof(buf));
int len = read(cfd, buf, sizeof(buf));
if(len > 0)
{
printf("Client say: %s\n", buf);
write(cfd, buf, len);
}
else if(len == 0)
{
printf("Client disconnected...\n");
}
else
{
perror("read");
}
return len;
}
In the example code above, the unused file descriptors are turned off in the parent-child process (the parent process does not need to communicate, and the child process does not need to listen). If the client disconnects actively, the server-side child process responsible for communicating with the client also exits. After the child process exits, it sends a signal called SIGCHLD to the parent process, which is captured by the sigaction() function in the parent process and recycled by waitpid() in the callback() function.
Another detail to explain is that this is the processing code of the parent process:
int cfd = accept(lfd, (struct sockaddr*)&cliaddr, &clilen);
while(1)
{
int cfd = accept(lfd, (struct sockaddr*)&cliaddr, &clilen);
if(cfd == -1)
{
if(errno == EINTR)
{
// The accept call was signaled off, unblocked, and returned -1
// Re-invoke accept once
continue;
}
perror("accept");
exit(0);
}
}
If the parent process calls the accept() function and does not detect a new client connection, the parent process is blocked here. At this time, the child process exits, signals the parent process, and the parent process captures the signal SIGCHLD. Because the signal has a high priority, it interrupts the normal execution of the code, so the parent process is blocked. Instead, the function callback() corresponding to the signal is processed, and then the function returns to accept () again, but this is no longer blocking. The function returns directly to -1, where the function call fails with the error description accept: Interrupted system call and the corresponding error number EINTR, because the code is an error caused by the signal interruption. So this error number can be judged in the program, and the parent process can call accept () again to continue blocking or accepting new connections from the client.
3. Multithreaded concurrency
Writing a multi-threaded version of the concurrent server program and multi-process thinking are similar, consider understanding the logo seating. There are two main types of threads in a multithread: the main thread (the parent thread) and the child thread, which handle the listening and communication processes on the server side, respectively. Based on the idea of multiprocess processing, it can be designed as follows:
Main thread:
- Responsible for listening and handling client connection requests, i.e. looping the accept() function in the parent process
- Create a sub-thread: Create a new connection and create a new sub-process to communicate with the corresponding client
- Recycle child thread resources: Since recycling requires calling blocking functions, which affects accept(), do thread detachment directly.
Subthread:
- Responsible for communication, file descriptors obtained after a new connection is made based on the main thread, and the corresponding client completes the receiving and sending of data.
- Send data: send() / write()
- Receive data: recv() / read()
In a multi-threaded server-side program, multiple threads share the same address space, some data is shared, some data is exclusive. Here are some details:
- Stack space for multiple threads in the same address space is exclusive
- Multiple threads share global data area, heap area, and file descriptors for the kernel area, so you need to be aware of data coverage issues and synchronize threads when multiple threads access shared resources.
The multithreaded version of the Tcp server sample code is as follows:
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <arpa/inet.h>
#include <pthread.h>
struct SockInfo
{
int fd; // Signal communication
pthread_t tid; // thread ID
struct sockaddr_in addr; // Address Information
};
struct SockInfo infos[128];
void* working(void* arg)
{
while(1)
{
struct SockInfo* info = (struct SockInfo*)arg;
// receive data
char buf[1024];
int ret = read(info->fd, buf, sizeof(buf));
if(ret == 0)
{
printf("Client has closed the connection...\n");
info->fd = -1;
break;
}
else if(ret == -1)
{
printf("Failed to receive data...\n");
info->fd = -1;
break;
}
else
{
write(info->fd, buf, strlen(buf)+1);
}
}
return NULL;
}
int main()
{
// 1. Create a socket for listening
int fd = socket(AF_INET, SOCK_STREAM, 0);
if(fd == -1)
{
perror("socket");
exit(0);
}
// 2. Binding
struct sockaddr_in addr;
addr.sin_family = AF_INET; // ipv4
addr.sin_port = htons(8989); // Byte order should be network byte order
addr.sin_addr.s_addr = INADDR_ANY; // == 0, IP acquisition is handed over to the kernel
int ret = bind(fd, (struct sockaddr*)&addr, sizeof(addr));
if(ret == -1)
{
perror("bind");
exit(0);
}
// 3. Set up monitoring
ret = listen(fd, 100);
if(ret == -1)
{
perror("listen");
exit(0);
}
// 4. Wait, accept connection request
int len = sizeof(struct sockaddr);
// Data Initialization
int max = sizeof(infos) / sizeof(infos[0]);
for(int i=0; i<max; ++i)
{
bzero(&infos[i], sizeof(infos[i]));
infos[i].fd = -1;
infos[i].tid = -1;
}
// Parent process listening, child process communication
while(1)
{
// Create Subthread
struct SockInfo* pinfo;
for(int i=0; i<max; ++i)
{
if(infos[i].fd == -1)
{
pinfo = &infos[i];
break;
}
if(i == max-1)
{
sleep(1);
i--;
}
}
int connfd = accept(fd, (struct sockaddr*)&pinfo->addr, &len);
printf("parent thread, connfd: %d\n", connfd);
if(connfd == -1)
{
perror("accept");
exit(0);
}
pinfo->fd = connfd;
pthread_create(&pinfo->tid, NULL, working, pinfo);
pthread_detach(pinfo->tid);
}
// Release Resources
close(fd); // Monitor
return 0;
}
When writing multithreaded concurrent server code, it is important to note that parent and child threads share file descriptors in the same address space, so each time a new connection is made in the main thread, the resulting file descriptor values need to be saved and cannot be overwritten on the same variable. This loses the previous file descriptor value and does not know how to communicate with the client.
In the example code above, the file descriptor values for communication obtained after a successful connection are saved in a global array. Each subthread needs to communicate with a different client and the required file descriptor values are different, as long as the variable that stores each valid file descriptor value corresponds to a different memory address. When using, data coverage will not occur, causing confusion of communication data.
IV. Technical Preparation
The above understanding requires a deeper understanding of multithreaded programming and string processing.
Multithreaded Programming Related Blogs
Blogs about C/C++ string handling should be coming soon!
ref:https://mp.weixin.qq.com/s/EN3UiTZbvuWEHPD8fxiVgQ