preface

Originally, I thought this project was very incompetent. After all, I didn't pass the stress test in the end.
I dare not submit my resume even when I submit it.

But after communicating with the counselor yesterday, what's embarrassing? Even if I'm working on a project now, it's difficult to meet people's expectations.
But what's the shame of having a sophomore or even junior non software engineering major design and write this project independently?

functional requirement

Let's take a look at the functional requirements first:

Module name	Function category	Subfunction
Local cloud client	System startup	Local cloud startup, initialization interface, software version and other initialization data synchronization
	User registration	The user enters the user name, password and security mobile phone number to register a new account
	User login	The user enters the user name and password, logs in to the server and obtains the list of documents stored by the user
	Change Password	The user enters the user name, old password and new password to complete the password modification
	Retrieve password	The user enters the user name and security phone, and sends it to the server to obtain the password
	File list	Get the list of files that should appear in the current folder
	File upload	Upload files to the specified folder
	File download	Download files from the specified folder
	Local download file management	The downloaded files shall be managed by the local folder
	Communication record	Upload and download records
	File sharing	Generate links and extraction codes to share files with other users
	System settings	Configure the ip address and port settings of the service
–	–	–
Local cloud server	User access	Receive client connections and manage client information
	Business processing	Handle various types of business of the client
	Heartbeat processing	Group customization
	file management	Manage the files transmitted by customers
	Database design and Application	Each group is customized to serve file management
	Concurrent load	Initialize the processing thread of 30
	journal	Generate an operation log every day, including business processing, operation status, fault records, etc
–	–	–
Pressure test procedure	Single customer single business test	It mainly tests the processing capacity and data accuracy of a single business of the server
	Multi customer and multi business	Test server throughput for different types of services
	Test report	Give key parameters and test results, such as business success number, failure number, business type, etc
	Pressure parameter setting	During each test, users can customize the test duration, business type, number of simulated clients, etc., and save them to the parameter file
	Test log	Record each test result and generate one every day

The following are non functional requirements:

Function category	Function description
10000 level concurrency	One server is required to be able to support more than 10000 connections
Large concurrent service	It is required that one server can withstand 200-300 concurrent business processing
Distributed architecture	TCP flow protocol is used for interprocess communication
System expansibility	Be able to expand business without modifying the source code
system safety	Suffer some illegal bombing

In fact, the main thing to do is these non functional requirements. The above functional requirements have been done. Just modify them slightly.

Data package (replaced by PB for subsequent projects)

First, of course, it is necessary to coordinate all parties and make a packet agreement.

#ifndef I_PACKET_PUBLIC_H
#define I_PACKET_PUBLIC_H

#include <string>
#include <iostream>

using namespace std;


#define MAX_LEN      256

/************Access layer packet header************/

typedef struct packet_header_st
{
    int fd;//Used for front and back end communication, i.e. target client fd (used by server)
    int funcId; // Function number
        //Login package 0x01, registration package 0x02, retrieve password 0x03, modify password 0x04

        //The client obtains the file list 0x11, uploads the file 0x12, downloads the file 0x13, and shares the file 0x14
        //Heartbeat 0x21

    int optid; // Operation code: request 0x00 and response 0x01

    int usrlenth;// Length of inclusion
    int packet_seq; //Serial number package
    int packet_sum; //Total number of packages

    char srcAddr[6]; //reserve
    char dstAddr[6]; //reserve

    int syn; // Determine whether the header is correct 0x04
}packet_header_t;

/************Access layer packet tail************/

typedef struct packet_tali_st//The tail of the packet is used to verify the integrity of the packet
{
    int pack_tail;//Set to 0x05
}packet_tali_t;

/************Overall packet************/

typedef struct  packet_all_st
{
    packet_header_t head;
    char body[packet_header_t::usrlenth];
    packet_tali_st tail;
    //unsigned len; // Effective length of data to be sent
}packet_all_st;


/************Business layer packet body************/

//Client login request package
typedef struct login
{
    int id;
    char psw[10];       //password
}Login_t;

//Login response package
typedef struct res_login_st
{
    int login_ret;  //Login result: 1 - login succeeded, 0 - login failed
}res_login_t;

//Client registration request package
typedef struct Register
{
    int id; //account number
    char tel[12];	//11 digit mobile number
    char psw[10];       //password
}Register_t;

//Register response package
typedef struct res_register_st
{
    int register_ret;	//Registration result: 0 - registration succeeded, 1 - registration failed
}res_register_t;

//Client modification password request package
typedef struct Change_PWD
{
    int id; //account number
    char psw[10];       //password
}c_pwd_t;

//Password change response package
typedef struct res_changepwd_st
{
    int change_ret;	//Password change result: 0 - password change succeeded, 1 - password change failed
}res_cpwd_t;

//Retrieve password request package
typedef struct findpwd
{
    int id;
    char tel[12];
}F_Pwd_t;

//Retrieve password response package
typedef struct res_findpwd_st
{
    int find_ret;	//Retrieval result: 0 - retrieval succeeded, 1 - retrieval failed
    char pwd[20];   //password
}res_fpwd_t;


/************Request business transaction flow request package************/

//Upload file request package
typedef struct UpFile
{
    int id;
}UpFile_t;

//Download video request package
typedef struct Download_File
{
    int user_id;    //User ID
    char *file_id;      //file name
}Download_t;

//File list request package
typedef struct File_List
{
    char *dir_name;	//Directory name
}FileList_t;

//File list response package
typedef struct Res_VideoList
{
    char V[800];
}res_filelist;

#endif // PACKET_BASE_H

However, it is inevitable that the data packet will not be modified in the future if it is determined too early. It was really disturbed by it before, until I contacted the PB protocol. I think this phenomenon can be effectively improved.

The database is configured separately

I used to like to put the initialization of the database into the main program. Later, when I bragged to my cousin who had been developing for N years, he said: why do you put the initialization of the database here? Too fast to boot? Not afraid of repeated initialization?

Oh, that makes sense. It seems to make sense. It does make sense.

Then I changed it. My database and other external dependencies that need to be initialized will be initialized separately!!!

In addition, the reason why I chose sqlite instead of MySQL and even used redis cautiously was that after communicating with another senior who was responsible for back-end development in the game company, the senior told me: you have limited the performance before development. Your data has to go through two layers of IO every time, not a lot of data. Do you know what is the most vulnerable in back-end development? It's not high load operation, it's io. IO is the most vulnerable.

OK, I'll change it. They are all predecessors. I'm still in the ivory tower. It's time to ask the senior students out for dinner.

The code is too long. It's only for early and late. Here's the sorting: FTP file management project (local cloud) project daily report (II)

Multi server interprocess communication

Mainly because I feel that if I want to join a new process in the future, the two connected services will be a little awkward. So I came up with such an idea.
However, the deficiencies are also obvious. If the service in the middle breaks down, there is no need to talk about everything, and the pressure of the service in the middle is also great (in fact, it's not much, it's just a transfer, and the middle doesn't have to be a service. It can be a cluster, which I didn't expect at that time)

The intermediary server (central control) adopts accept, and all edge servers adopt connect. After the connection is successful, I report my situation to the central control center (what is the server name corresponding to fd, which is convenient for communication). However, my talent is stupid. I thought about this picture all night.

epoll module

Responsibility chain model.

Details: Daily report of FTP file management project (local cloud) (VI)

So far, I still have a lot of diagrams, and the code has been written. You can implement it yourself, and then we can discuss it together.