1, fasdfs communication protocol

Official reference: https://mp.weixin.qq.com/s/lpWEv3NCLkfKmtzKJ5lGzQ

FastDFS adopts binary TCP communication protocol. A packet consists of a header and a body. The header has only 10 bytes, and the format is as follows:

@ pkg_len: 8 Byte integer, body Length, excluding header，just body Length of
@ cmd: 1 Byte integer, command code
@ status: 1 Byte integer, status code, 0 indicates success, non-0 indicates failure( UNIX Error code)

The code is as follows. We can see that the cmd author uses char type, and char has only one byte,
The char type is also a standard integer type. However, the type name char with only one word can be synonymous with either signed char or unsigned char, which is determined by the compiler. Because this is selected by the implementation version adopted, strictly speaking, char, signed char and unsigned char are three different data types.

If the char value used by the program includes cases less than 0 or greater than 127, you should use signed char or unsigned char instead of char.

Common storage space size and value range of standard integer types
Type storage size min max
Char (same as signed char or unsigned char)
unsigned char 1 byte 0 255
signed char 1 byte - 128 127

typedef struct
{
	char pkg_len[FDFS_PROTO_PKG_LEN_SIZE];  //body length, not including header
	char cmd;    //command code
	char status; //status code for response
} TrackerHeader;

Therefore, the author's instruction codes are between 0-127.

Description of the type in the packet:

1) Integer type adopts network byte order (big endian), including 4-byte integer and 8-byte integer;

2) There is no byte order problem for 1-byte integers, which are directly mapped to byte type in Java and char type in C/C + +;

3) The fixed length string type ends with ASCII code 0. For languages such as Java, you need to call trim to process the returned string. The length of variable length string can be obtained directly or calculated according to the package length, and does not end with ASCII 0.

tracker_ All command codes are defined in proto. H.

The following lists the command codes sent by the client to the FastDFS server and their body structure.

1. Public command code

• FDFS_PROTO_CMD_ACTIVE_TEST: activation test, usually used to detect whether the connection is valid. When the client uses the connection pool, it can maintain a long connection with the server by sending an active test after establishing a connection.

  #Request body: None
  #Response body: None

2. Command code sent to storage server

• STORAGE_PROTO_CMD_UPLOAD_FILE: upload ordinary files

• STORAGE_PROTO_CMD_UPLOAD_APPENDER_FILE: upload an appender type file

  # Request body:

      @store_path_index: 1 Byte integer, storage path sequence number based on 0 

      @meta_data_length: 8 Byte integer, meta data(File attachment attribute) length, which can be 0
      @file_size: 8 Byte integer, file size

     @file_ext_name: 6 Byte string, file extension without decimal point, for example jpeg,tar.gz

      @meta_data:  meta_data_length Byte string, file additional attributes, each attribute is in characters \x01 Separator, name key Sum value value Use characters between \x02 separate
      @file content: file_size Byte binary content 
    # Response body:
      @ group_name: 16 Byte string, group name 
      @ filename: Variable length string, file name

• STORAGE_PROTO_CMD_UPLOAD_SLAVE_FILE: upload the slave file

  # Request body:

      @master_filename_length: 8 Byte integer, length of main file name 

      @meta_data_length: 8 Byte integer, meta data(File attachment attribute) length, which can be 0
      @file_size: 8 Byte integer, file size

      @filename_prefix: 16 Byte string from file prefix name

      @file_ext_name: 6 Byte string, file extension without decimal point, for example jpeg,tar.gz

      @master_filename: master_filename_length Byte string, main file name

      @meta_data: meta_data_length Byte string, file additional attributes, characters for each attribute record \x01 Separator, name key Sum value value Use characters between \x02 separate
      @file content: file_size Byte binary content 
    # Response body:
      @ group_name: 16 Byte string, group name
      @ filename: Variable length string, file

• STORAGE_PROTO_CMD_DELETE_FILE: delete files

 # Request body:
      @group_name: 16 Byte string, group name 
      @filename: Variable length string, file name
  # Response body: None

• STORAGE_PROTO_CMD_SET_METADATA: set meta data (file attachment attribute)

   # Request body:

     @filename_length: 8 Byte integer, filename length

     @meta_data_length: 8 Byte integer, meta data(File attachment attribute) length, which can be 0   

      @op_flag: 1 Byte character, operation mark, value description is as follows:

           'O' - Covering method meta data
           'M' - merge Mode, and original meta data Merged together, existing properties will be overwritten
      @group_name: 16 Byte string, group name 

      @filename: filename_length Byte string, file name
      @meta_data: meta_data_length Byte string, file additional attributes, characters for each attribute record \x01 Separator, name key Sum value value Use characters between \x02 separate
  # Response body: None

• STORAGE_PROTO_CMD_DOWNLOAD_FILE: Download File

  # Request body:
      @file_offset: 8 Byte integer, file offset

      @download_bytes: 8 Byte integer, number of bytes downloaded

      @group name: 16 Byte string, group name 
      @filename: Variable length string, file name
  # Response body:
      @file_content: Variable length binary content   
 
 * STORAGE_PROTO_CMD_GET_METADATA: obtain meta data(File (additional properties)
  # Request body:
      @group name: 16 Byte string, group name
      @filename: Variable length string, file name
  # Response body:
      @meta_data: Variable length string, file additional attributes, and characters for each attribute record \x01 Separator, name key Sum value value Use characters between \x02 separate

• STORAGE_PROTO_CMD_QUERY_FILE_INFO: get file information

  # Request body:
      @group name: 16 Byte string, group name
      @filename: Variable length string, file name
  # Response body:

      @file_size: 8 Byte integer, file size

      @create_timestamp: 8 Byte integer, file creation time( Unix Time stamp)

      @crc32: 8 Byte integer, file content CRC32 Check code

      @source_ip_addr: 16 Byte string, source storage server IP address

• STORAGE_PROTO_CMD_APPEND_FILE: append file content

  # Request body:
      @appender_filename_length: 8 Byte integer, appender File name length

      @file_size: 8 Byte integer, file size

      @appender_filename: appender_filename_length Byte digit string, appender file name

      @file_content: file_size Binary content of bytes, additional file content 

  # Response body: None

• STORAGE_PROTO_CMD_MODIFY_FILE: modify file content

  # Request body:
      @appender_filename_length: 8 Byte integer, appender File name length

      @file_offset: 8 Byte integer, file offset

      @file_size: 8 Byte integer, file size

      @appender_filename: appender_filename_length Byte digit string, appender file name

      @file_content: file_size Binary content of bytes, new (target) file content 

  # Response body: None

• STORAGE_PROTO_CMD_TRUNCATE_FILE: change file size

  # Request body:
      @appender_filename_length: 8 Byte integer, appender File name length

      @truncated_file_size: 8 Byte integer, truncate File size after

      @appender_filename: appender_filename_length Byte digit string, appender file name

  # Response body: None

• STORAGE_PROTO_CMD_REGENERATE_APPENDER_FILENAME: rename an appender type file to a normal file

  # Request body:

      @appender_filename: Variable length string, appender file name

  # Response body:

      @group name: 16 Byte string, group name 
      @filename: Variable length string, regenerated file name (normal type)

2, fastdfs storage custom script

1. Customize the command code of storage server: calculation file md5

Overall idea: involving file storage_ Define command codes and custom md5 functions in service. C.
Calculate md5 imitation file download storage_server_download_file code is to calculate md5 while reading.

Because the author of the command code uses char type, the use range of numbers is limited to 0 ~ 127. Observe the author's use, tracker_proto.h, as of version 6.06, only 50-60 and 120-127 are unused. Here I define it as 120.

1. storage_ Define user-defined codes and user-defined function calls in service. C
   Note: for subsequent fastdfs code upgrades, pay attention to whether the command codes defined by you conflict.

#define STORAGE_PROTO_CMD_QUERY_FILE_MD5		       120

In int storage_ deal_ Add your command code case at the end of task (struct fast_task_info * ptask)
As follows, it is recommended to add it to the front, so that the official upgrade can be good diff

	switch(pHeader->cmd)
	{
        case STORAGE_PROTO_CMD_QUERY_FILE_MD5:
            ACCESS_LOG_INIT_FIELDS();
            result = storage_server_query_file_md5_test(pTask);
            STORAGE_ACCESS_LOG(pTask, \
				ACCESS_LOG_ACTION_QUERY_FILE, \
				result);
            break;

1. storage_ Write a custom function storage in service. C_ server_ query_ file_ md5_ Test imitation storage_server_download_file function logic

This function is mainly processed and then adjusted to torage_. Our custom function in DIO. C.

1. storage_dio.c is used to write io related operations. Here we define the calculation md5 logic for reading files.
   Here are the specific open files fc_safe_read reads the byte stream.

3, Reference

C language integer type (including value range and length)
Reference URL: http://c.biancheng.net/view/177.html