minio cluster installation document

Posted by terry1989 on Thu, 10 Feb 2022 18:45:10 +0100

1. Preparatory work

1. min officials recommend at least 4 nodes

nodeIPdata
youduk2xxx.xxx.xxx.xxx/home/minio/data1 /home/minio/data2
youduk3xxx.xxx.xxx.xxx/home/minio/data1 /home/minio/data2
youduk4xxx.xxx.xxx.xxx/home/minio/data1 /home/minio/data2
youduk5xxx.xxx.xxx.xxx/home/minio/data1 /home/minio/data2

2,mkdir /home/minio/{data1,data2,run}

Create data1,data2 folders. Create the startup script run folder.

3. Upload the binary file miniio in the directory to / home/minio/

4. Modify the maximum number of files that the server can read

/etc/security/limits.conf

soft    nofile  65535
hard    nofile  65535

2. Script configuration

1. Configure the startup script.

  • MINIO_ACCESS_KEY: user name, with a minimum length of 5 characters
  • MINIO_SECRET_KEY: password. The password cannot be set too simply, or minio will fail to start. The minimum length is 8 characters
  • – config dir: Specifies the directory of the cluster configuration file
#!/bin/bash
export MINIO_ACCESS_KEY=admin
export MINIO_SECRET_KEY=admin456789
/home/minio/minio server --config-dir /etc/minio \
--address "youduk2:9029" \
http://youduk2/home/minio/data1 http://youduk2/home/minio/data2 \
http://youduk3/home/minio/data1 http://youduk3/home/minio/data2 \
http://youduk4/home/minio/data1 http://youduk4/home/minio/data2 \
http://youduk5/home/minio/data1 http://youduk5/home/minio/data2

Minio has a default 9000 port. Add – address "youduk2:9029" in the configuration file to change the port

MINIO_ACCESS_KEY: user name, with a minimum length of 5 characters
MINIO_SECRET_KEY: password. The password cannot be set too simply, or minio will fail to start. The minimum length is 8 characters

– config dir: Specifies the directory of the cluster configuration file

2. Create Minio server

vim /usr/lib/systemd/system/minio.service

[Unit]
Description=Minio service
Documentation=https://docs.minio.io/
[Service]
WorkingDirectory=/home/minio/run
ExecStart=/home/minio/run/run.sh
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target

3. Modify file permissions

chmod +x /usr/lib/systemd/system/minio.service && chmod +x /home/minio/minio && chmod +x /home/minio/run/run.sh

4. Start cluster

systemctl daemon-reload

systemctl start minio

systemctl enable minio

View cluster status

systemctl status minio.service

5. Start other nodes

youduk3, youduk4 and youduk5 perform the 2 and 3 operations above

youduk3, youduk4 and youduk5 perform the operations 1, 2 and 3 of the above installation configuration

youduk3, youduk4, youduk5 execution

sudo systemctl start minio.service
sudo systemctl status minio.service
journalctl -f -u minio.service

Test youduk3:9029

Page request succeeded.

Enter the username and password admin admin456789

3. Configure installation

Environmental preparation

Servers: youduk1, youduke2, youduk3, youduk4

File: / home/minio/data/{data1,data2,data3,data4}

Profile installation

vim /usr/lib/systemd/system/minio.service

[Unit]
Description=MinIO
Documentation=https://docs.min.io
Wants=network-online.target
After=network-online.target
#Binary file location
AssertFileIsExecutable=/home/minio/minio
[Service]
WorkingDirectory=/home/minio/
User=root
Group=root
ProtectProc=invisible
#Configure the environment address.
EnvironmentFile=-/home/minio/run/minio
#ExecStartPre=/bin/bash -c "if [ -z \"${MINIO_VOLUMES}\" ]; then echo \"Variable MINIO_VOLUMES not set in /home/minio/run/minio\"; exit 1; fi"
ExecStart=/home/minio/minio server $MINIO_OPTS $MINIO_VOLUMES
# Set startup
# Let systemd restart this service always
Restart=always

#Set the maximum number of open files
# Specifies the maximum file descriptor number that can be opened by this process
LimitNOFILE=65536
# Specify the maximum thrjournalctl - F - u Minio that this process can create Number of serviceeads
# Specifies the maximum number of thrjournalctl -f -u minio.serviceeads this process can create
TasksMax=infinity
# Disable the timeout logic and wait for the process to stop
# Disable timeout logic and wait until process is stopped
TimeoutStopSec=infinity
SendSIGKILL=no

[Install]
WantedBy=multi-user.target

chmod +x /usr/lib/systemd/system/minio.service

Each node

/home/minio/run/minio file configuration

# Set the hosts and volumes MinIO uses at startup
# The command uses MinIO expansion notation {x...y} to denote a
# sequential series.
#
# The following example covers four MinIO hosts
# with 4 drives each at the specified hostname and drive locations.

#MINIO_VOLUMES="http://youduk2/home/minio/data1 http://youduk2/home/minio/data2 http://youduk3/home/minio/data1 http://youduk3/home/minio/data2 http://youduk4/home/minio/data1 http://youduk4/home/minio/data2 http://youduk5/home/minio/data1 http://youduk5/home/minio/data2"
MINIO_VOLUMES="http://youduk{1...4}:9029/home/minio/data/data{1...4}"
# Set all MinIO server options
#
# The following explicitly sets the MinIO Console listen address to
# port 9001 on all network interfaces. The default behavior is dynamic
# port selection.

#MINIO_OPTS="--console-address :9001 --address youduk3:9029 --config-dir /etc/minio"
MINIO_OPTS="--console-address :9001  --address youduk2:9029"  #--address youduk2:9029 write local domain name: Port
# Set the root username. This user has unrestricted permissions to
# perform S3 and administrative API operations on any resource in the
# deployment.
#
# Defer to your organizations requirements for superadmin user name.

MINIO_ROOT_USER=amdin

# Set the root password
#
# Use a long, random, unique string that meets your organizations
# requirements for passwords.

MINIO_ROOT_PASSWORD=admin456789

# Set to the URL of the load balancer for the MinIO deployment
# This value *must* match across all MinIO servers. If you do
# not have a load balancer, set this value to to any *one* of the
# MinIO hosts in the deployment as a temporary measure.
# MINIO_SERVER_URL="http://youduk"

Each node should be configured with MINIO_OPTS --address youduk2:9029 modify the domain name of the cost machine corresponding to the server

The browser access address is 9001, and the cluster development port is 9029. Youduk {1... 4} is a server pool

Software deployment per node

Upload binary files to / home/minio /

chmod +x /home/minio/minio

The youduk2, youduk3, youduk4 and youduk5 nodes are started respectively:

systemctl daemon-reload

systemctl start minio

systemctl enable minio

Test address http://youduk2:9001

Test address http://youduk4:9001

Login user name - password

Test successful

4. Cluster expansion

MinIO supports two expansion modes:

  • Expand the cluster by modifying the command line and specifying a new cluster set on the command line
  • By introducing the third-party component etcd, the dynamic expansion scheme is realized on the basis of the original cluster

1. Configure extension mode

MinIO supports the expansion of distributed clusters by specifying a new cluster set on the command line. The storage capacity of multiple nodes is the storage capacity of distributed MinIO.

The deployment has a server pool consisting of four MinIO server hosts with sequential host names.

youduk1   youduk2
youduk3   youduk4

Each host has four locally connected drives with sequential mount points: I have no nodes. Test only directories

/home/minio/data/data1   /home/minio/data/data2
/home/minio/data/data3   /home/minio/data/data4

The new server pool consists of eight new MinIO hosts with sequential host names:

youduk5   youduk6  youduk7   youduk8
youduk9   youduk10  youduk11   youduk12

All new hosts have eight locally attached disks with sequential mount points:

/home/minio/data/data1   /home/minio/data/data2
/home/minio/data/data3   /home/minio/data/data4
/home/minio/data/data5   /home/minio/data/data6
/home/minio/data/data7   /home/minio/data/data8

The environment configuration is installed in the same way as the above configuration

Each node

MINIO_VOLUMES=“ http://youduk{1... 4}: "9029 / home / Minio / data / data {1... 4}" is modified to

MINIO_VOLUMES="http://youduk{1...4}:9029/home/minio/data/data{1...4} http://youduk{5...12}:9029/home/minio/data/data{1...8}"

‚Äč

Through the above expansion strategy, the cluster can be expanded on demand. Restarting the cluster after reconfiguration will take effect immediately in the cluster and have no impact on the existing cluster. In the above command, we can regard the original cluster as a cluster pool and the new cluster as another cluster pool. The new objects are placed in the cluster pool according to the proportion of available space in each cluster pool. Within each hash pool, the location is determined based on the hash algorithm.

Note: each added cluster pool must have the same number of disks (erasure code set) size as the original cluster pool in order to maintain the same data redundancy SLA. For example, if the first cluster pool has 8 disks, you can expand the cluster to a cluster pool of 16, 32 or 1024 disks. You only need to ensure that the deployed SLA is a multiple of the original cluster pool.

2. etcd extension scheme

etcd for bucket DNS service records

reference resources https://www.sohu.com/a/455702322_115128

Topics: Java