1, FastDFS components and other nouns

1,tracker server

Tracking server: used to schedule requests from clients. And record the information status of all storage groups and storage servers in memory.

2,storage server

Storage server: used to store files (data) and file attributes (metadata)

3,client

Client: the initiator of the service request, which interacts with the tracker and storage server based on TCP protocol through a special interface

group

Group, also known as volume: the files in the same group are identical

Document identification

It includes two parts: group name and file name (including path)

meta data

Key pair file: value related attribute

fid

File identifier:

2.1 environment:

Server version: CentOS release 6.3 (Final)

Kernel version: Linux YUNWEI_BIGDATA_SERVER_02 2.6.32-279.el6.x86_64 #1 SMP Fri Jun 22 12:19:21 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

fdfs version: FastDFS_v4.05.tar.gz,libfastcommon_v1.01.tar.gz

2.3 compiling libfastcommon

    cd fast-dep

tar zxf libfastcommon_v1.01.tar.gz

cd libfastcommon-1.0.1

./make.sh

./make.sh install

2.4 compiling FastDFS

cd   /home/isvision/fastdfs-dep/

tar -zxvf  FastDFS_v4.05.tar.gz         

cd FastDFS

./make.sh

Here if Maike The SH execution reports an error. Note that there may be a lack of libevent devel, as shown below:

tracker_service.c:253: error: storage size of 'ev_notify' isn't known
tracker_service.c:258: warning: implicit declaration of function 'event_set'
tracker_service.c:259: error: 'EV_READ' undeclared (first use in this function)
tracker_service.c:259: error: (Each undeclared identifier is reported only once
tracker_service.c:259: error: for each function it appears in.)
tracker_service.c:259: error: 'EV_PERSIST' undeclared (first use in this function)
tracker_service.c:260: warning: implicit declaration of function 'event_base_set'
tracker_service.c:266: warning: implicit declaration of function 'event_add'
tracker_service.c:275: warning: implicit declaration of function 'event_base_loop'
tracker_service.c:279: warning: implicit declaration of function 'event_base_free'
tracker_service.c:253: warning: unused variable 'ev_notify'
make: *** [tracker_service.o] Error 1

Fix it after installing libevent devel.

Continue execution

./make.sh install

2.5 inspection and installation

fdfs_storage,fdfs_trackerd,fdfs_ The presence of the monitor command indicates that the fastdfs installation has been completed. Enter FDFS on the command line and press tab to automatically jump out.

3. Disposition

3.1 ， create folders required for runtime

Folders to be created in advance on the Tracker server:

mkdir  /data/server/fastdfs/{tracker1,tracker2...}

/ / generally, the monitor service and the tracker are installed on the same machine, so fastdfs_ The client folder is also created on the tracker server.

Pre created folders on the Storage server:

mkdir  /data/server/fastdfs/{storage1,storage2...}

I'm here for more track s and more storage, so I don't give examples one by one.

3.2. Modify configuration file

#step 1. Copy all configuration files under FastDFS/conf / to the path / etc/fdfs /

[root@dzytest FastDFS]# cp -R /home/isvision/fastdfs-dep/FastDFS/conf /*  /etc/fdfs/

#step 2. Modify the configuration file and modify the corresponding conf according to the role of the machine. For example, the tracker machine only needs to modify the tracker Conf, the storage machine only needs to modify the storage conf. The Monitor role is a monitoring device that can be placed on any machine in the FastDFS cluster (fdfs_monitor is required), usually on the tracker machine.

Tracker:

disabled=false
bind_addr=
port=22122
connect_timeout=30
network_timeout=60
base_path=/data/server/fastdfs/tracker1
max_connections=10000
accept_threads=1
work_threads=10
min_buff_size = 8KB
max_buff_size = 128KB
store_lookup=0
store_group=group1
store_server=2
store_path=0
download_server=0
reserved_storage_space = 10%
log_level=info
run_by_group=yw
run_by_user=dhom
allow_hosts=*
sync_log_buff_interval = 10
check_active_interval = 120
thread_stack_size = 64KB
storage_ip_changed_auto_adjust = true
storage_sync_file_max_delay = 86400
storage_sync_file_max_time = 300
use_trunk_file = false 
slot_min_size = 256
slot_max_size = 16MB
trunk_file_size = 64MB
trunk_create_file_advance = false
trunk_create_file_time_base = 02:00
trunk_create_file_interval = 86400
trunk_create_file_space_threshold = 20G
trunk_init_check_occupying = false
trunk_init_reload_from_binlog = false
trunk_compress_binlog_min_interval = 0
use_storage_id = false
storage_ids_filename = storage_ids.conf
id_type_in_filename = ip
store_slave_file_use_link = false
rotate_error_log = false
error_log_rotate_time=00:00
rotate_error_log_size = 0
log_file_keep_days = 0
use_connection_pool = false
connection_pool_max_idle_time = 3600
http.server_port=8080
http.check_alive_interval=30
http.check_alive_type=tcp
http.check_alive_uri=/status.html

group_name = group name (hint: the name of the group name should be composed of English and data as much as possible, such as group0,group1)

tracker_server=IP:22122 (prompt: Please enter the address where the tracker server is deployed)

storage:

disabled=false
group_name=group1
bind_addr=0.0.0.0
client_bind=true
port=23001
connect_timeout=30
network_timeout=60
heart_beat_interval=30
stat_report_interval=60
base_path=/data/server/fastdfs/storage1
max_connections=10000
buff_size = 256KB
accept_threads=1
work_threads=4
disk_rw_separated = true
disk_reader_threads = 1
disk_writer_threads = 1
sync_wait_msec=50
sync_interval=0
sync_start_time=00:00
sync_end_time=23:59
write_mark_file_freq=500
store_path_count=1
store_path0=/data/server/fastdfs/storage1
subdir_count_per_path=256
tracker_server=192.168.165.13:22122   #Internal and external IP address of track
tracker_server=192.168.165.2:22122    #Internal and external IP address of track
log_level=debug
run_by_group=
run_by_user=
allow_hosts=*
file_distribute_path_mode=0
file_distribute_rotate_count=100
fsync_after_written_bytes=0
sync_log_buff_interval=10
sync_binlog_buff_interval=10
sync_stat_file_interval=300
thread_stack_size=512KB
upload_priority=10
if_alias_prefix=
check_file_duplicate=0
file_signature_method=hash
key_namespace=FastDFS
keep_alive=0
use_access_log = false
rotate_access_log = false
access_log_rotate_time=00:00
rotate_error_log = false
error_log_rotate_time=00:00
rotate_access_log_size = 0
rotate_error_log_size = 0
log_file_keep_days = 0
file_sync_skip_invalid_record=false
use_connection_pool = false
connection_pool_max_idle_time = 3600
http.domain_name=
http.server_port=8888

4. Run

4.1 # startup procedure

Ø start the corresponding program according to the machine role (the service starts smoothly, but start the tracker first and then the storage)

tracker program startup command:

fdfs_trackerd /etc/fdfs/tracker.conf

storage program startup command:

fdfs_storaged /etc/fdfs/storage.conf

Ø check whether the program is started

Just check the port and process running status

 

View running status

You can view the status of the whole FastDFS system through monitor:

fdfs_monitor /etc/fdfs/client.conf

View the status of a single server

ssh name@ip

tracker:

cat /data/server/fastdfs/tracker1/logs/trackerd.log

storage: 

cat /data/server/fastdfs/storage1/logs/storaged.log

For example:

5. Testing

Method 2: (please execute on the deployment tracker server!)

First configure / etc / FDFS / client Conf file and modify the tracker server ip in it.

Test with the following command:

fdfs_upload_file /etc/fdfs/client.conf file to upload

For example:

fdfs_upload_file /etc/fdfs/client.conf  /home/isvision/fastdfs-dep.zip

Results returned:

group0/M00/00/00/wKhztFXKrV-APnOHABPE0v0bfWo251.zip

Address of actual data:

/data/server/fastdfs/storage1/data/00/00

Backend nginx configuration:

server {

listen 2000;

server_name 192.168.12.28;

access_log /data/logs/nginx/access.end.log main;

error_log /data/logs/nginx/error.end.log;

location /group1/M00/ {

alias /data/server/fastdfs/storage1/data/;

}

location ~ /group1/M00/(.+)_(\d+)x(\d+)\.(.*)$ {

set $w $2;

set $h $3;

rewrite /group1/M00/(.+)_(\d+)x(\d+)\.(.*)$ /img1/$1.$4 last;

}

location /img1/ {

alias /data/server/fastdfs/storage1/data/;

image_filter resize $w $h;

image_filter_buffer 5M;

}

location /group2/M00/ {

alias /data/server/fastdfs/storage2/data/;

}

location ~ /group2/M00/(.+)_(\d+)x(\d+)\.(.*)$ {

set $w $2;

set $h $3;

rewrite /group2/M00/(.+)_(\d+)x(\d+)\.(.*)$ /img2/$1.$4 last;

}

.

Troubleshooting 1:

2017-12-20 16:45:39] INFO - local_host_ip_count: 2, 127.0.0.1 172.18.126.33 

[2017-12-20 16:45:49] INFO - file: tracker_relationship.c, line: 383, selecting leader... 

[2017-12-20 16:45:58] INFO - file: tracker_relationship.c, line: 383, selecting leader... 

[2017-12-20 16:46:02] INFO - file: tracker_relationship.c, line: 383, selecting leader... 

[2017-12-20 16:46:06] INFO - file: tracker_relationship.c, line: 383, selecting leader... 

[2017-12-20 16:46:14] INFO - file: tracker_relationship.c, line: 383, selecting leader... 

[2017-12-20 16:46:15] INFO - file: tracker_relationship.c, line: 383, selecting leader... 

[2017-12-20 16:46:16] INFO - file: tracker_relationship.c, line: 383, selecting leader... 

[2017-12-20 16:46:26] INFO - file: tracker_relationship.c, line: 383, selecting leader... 

[2017-12-20 16:46:35] INFO - file: tracker_relationship.c, line: 383, selecting leader... 

[2017-12-20 16:46:38] INFO - file: tracker_relationship.c, line: 383, selecting leader... 

[2017-12-20 16:46:42] INFO - file: tracker_relationship.c, line: 383, selecting leader...

track has been voting and can't find the leader

Solution:

Stop all storage and restart all track s

Troubleshooting 2:

status 5 != 0

[2017-03-23 13:49:13] INFO - file: tracker_mem.c, line: 4213, sys files loaded from tracker server 192.168.10.78:22122

[2017-03-23 13:49:13] ERROR - file: tracker_mem.c, line: 596, in the file "/fastdfs/tracker/data/storage_groups_new.dat", item "group_count" is not found

[2017-03-23 13:49:13] ERROR - file: tracker_mem.c, line: 4297, get sys files from other trackers fail, errno: 2

[2017-03-23 13:49:19] ERROR - file: tracker_proto.c, line: 48, server: 192.168.10.78:22122, response status 5 != 0

[2017-03-23 13:49:19] INFO - file: tracker_mem.c, line: 4213, sys files loaded from tracker server 192.168.10.78:22122

[2017-03-23 13:49:19] ERROR - file: tracker_mem.c, line: 596, in the file "/fastdfs/tracker/data/storage_groups_new.dat", item "group_count" is not found

[2017-03-23 13:49:19] ERROR - file: tracker_mem.c, line: 4297, get sys files from other trackers fail, errno: 2

[2017-03-23 13:49:20] ERROR - file: tracker_proto.c, line: 48, server: 192.168.10.78:22122, response status 5 != 0

[2017-03-23 13:49:20] INFO - file: tracker_mem.c, line: 4213, sys files loaded from tracker server 192.168.10.78:22122

[2017-03-23 13:49:20] ERROR - file: tracker_mem.c, line: 596, in the file "/fastdfs/tracker/data/storage_groups_new.dat", item "group_count" is not found

[2017-03-23 13:49:20] ERROR - file: tracker_mem.c, line: 4297, get sys files from other trackers fail, errno: 2

.

Solution:

Because the data of multiple tracker s are inconsistent, there is no automatic synchronization.

Delete the / data directory of the newly added node and copy the / data file under the current Leader tracker directory in the past (. pid do not copy)