Linux learning – disk quota, VDO, virtual data optimization
Disk capacity quota
Soft limit: when the soft limit is reached, the user will be prompted. When the user is still allowed to continue to use within the limited limit
Hard limit: when the hard limit is reached, the user will be prompted and the user's operation will be forcibly terminated
The quota disk capacity quota service package comes with RHEL8, but it takes effect after you manually configure and restart the system.
Take the boot directory / boot as an example
1, Start disk quota
[root@localhost ~]# vim /etc/fstab 3 # /etc/fstab 4 # Created by anaconda on Wed Mar 17 00:16:10 2021 5 # 6 # Accessible filesystems, by reference, are maintained under '/dev/disk/'. 7 # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 8 # 9 # After editing this file, run 'systemctl daemon-reload' to update systemd 10 # units generated from this file. 11 # 12 /dev/mapper/rhel-root / xfs defaults 0 0 13 UUID=72e9e1b7-3326-413e-956f-d2ee7e41ad4a /boot xfs defaults,uquota 0 0 14 /dev/mapper/rhel-swap swap swap defaults 0 0 [root@localhost ~]# mount | grep boot /dev/nvme0n1p1 on /boot type xfs (rw,relatime,seclabel,attr2,inode64,usrquota)
2, Create an account and set write permissions on / boot
[root@localhost ~]# useradd zhangsan [root@localhost ~]# chmod -R o+w /boot
3, Configure disk quota
xfs_ The quota command is used to manage the disk capacity quota of the device. The syntax format is "xfs_quota [parameter] quota file system"
-c: Sets the command to execute as a parameter
-x: The expert mode enables the operation and maintenance personnel to configure the quota more complex
isoft and ihard limit the number of files by limiting the maximum number of inode s used by the system
bsoft and bhard represent the block size occupied by the file, that is, the maximum capacity occupied by the file
Soft is a soft limit. When the limit is exceeded, it only writes the operation records to the log without limiting the user's behavior. Hard is a hard limit. Once it exceeds the system, it will be banned immediately. Users can no longer create or occupy any new hard disk capacity.
[root@localhost ~]# xfs_quota -x -c 'limit bsoft=3m bhard=6m isoft=3 ihard=6 zhangsan' /boot
[root@localhost ~]# xfs_quota -x -c report /boot
User quota on /boot (/dev/nvme0n1p1)
Blocks
User ID Used Soft Hard Warn/Grace
---------- --------------------------------------------------
root 132452 0 0 00 [--------]
zhangsan 0 3072 6144 00 [--------]
4, Testing
Switch to the zhangsan account and create a 5MB and 8MB file test respectively
[root@localhost ~]# su - zhangsan [zhangsan@localhost ~]$ cd /boot [zhangsan@localhost boot]$ dd if=/dev/zero of=/boot/zhangsan bs=5M count=1 1+0 records in 1+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.00846127 s, 620 MB/s [zhangsan@localhost boot]$ dd if=/dev/zero of=/boot/zhangsan bs=8M count=1 dd: error writing '/boot/zhangsan': Disk quota exceeded 1+0 records in 0+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.0102364 s, 410 MB/s
It is found that a file of 5M size can be created, but you will be prompted when creating a file of 8M size
Disk quota exceeded, now increase the limit value of disk quota
5, Modify the value of the disk quota
The edquota command is used to manage the disk quota of the system. Its full English name is "edit quota", and its syntax format is "edquota [parameter] user name"
| parameter | effect |
|---|---|
| -u | Set up a user |
| -g | Set up a user |
| -p | Copy the original rule to the new user / group |
| -t | Limit grace period |
Change the value of the disk quota to 8M, and then create an 8M file test
[zhangsan@localhost ~]$ edquota -u zhangsan Disk quotas for user zhangsan (uid 1001): Filesystem blocks soft hard inodes soft hard /dev/nvme0n1p1 8192 3072 8192 1 3 6 [zhangsan@localhost ~]$ dd if=/dev/zero of=/boot/zhangsan bs=8M count=1 1+0 records in 1+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.00722449 s, 1.2 GB/s
After increasing the value of disk quota, the file was created successfully
VDO virtual data optimization
VDO (virtual data optimization) is a technology that optimizes storage space by compressing or deleting data on storage devices.
Red hat recommends that when deploying virtual machines or containers, it is recommended to use the ratio of logical storage to physical storage of 10:1, that is, 1TB physical storage corresponds to 10TB logical storage; When deploying object storage (such as Ceph), the ratio of logical storage to physical storage is 3:1, that is, 1TB physical storage corresponds to 3TB logical storage.
1, Add a hard disk first
[root@localhost ~]# fdisk -l Disk /dev/nvme0n2: 20 GiB, 21474836480 bytes, 41943040 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes
2, Install VDO related files
RHEL8 enables VDO by default
[root@localhost ~]# dnf install kmod-kvdo vdo Updating Subscription Management repositories. Unable to read consumer identity This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register. Last metadata expiration check: 1:35:25 ago on Tue 19 Oct 2021 11:03:26 PM CST. Package kmod-kvdo-6.2.0.293-50.el8.x86_64 is already installed. Package vdo-6.2.0.293-10.el8.x86_64 is already installed. Dependencies resolved. Nothing to do. Complete!
2, Create a VDO volume
Use the vdo command to add, where the name parameter represents the name of the new device volume; The device parameter represents which disk is made; The vdoLogicalSize parameter represents the size of the device after production. According to the principle recommended by red hat, 20GB hard disk will be turned into 200GB logical storage:
[root@localhost ~]# vdo create --name=sotrage --device=/dev/nvme0n2 --vdoLogicalSize=200G Creating VDO sotrage Starting VDO sotrage Starting compression on VDO sotrage VDO instance 0 volume is ready at /dev/mapper/sotrage
3, View the details of the new volume
The newly created volume is stored in the / dev/mapper directory
[root@localhost ~]# vdo status --name=sotrage
VDO status:
Date: '2021-10-20 00:41:43+08:00'
Node: localhost.localdomain
Kernel module:
Loaded: true
Name: kvdo
Version information:
kvdo version: 6.2.0.293
Configuration:
File: /etc/vdoconf.yml
Last modified: '2021-10-20 00:40:59'
VDOs:
sotrage:
Acknowledgement threads: 1
Activate: enabled
Bio rotation interval: 64
Bio submission threads: 4
Block map cache size: 128M
Block map period: 16380
Block size: 4096
CPU-work threads: 2
Compression: enabled
Configured write policy: auto
Deduplication: enabled
Device mapper status: 0 419430400 vdo /dev/nvme0n2 normal - online online 1051408 5242880
......
4, Format and mount the new volume
[root@localhost ~]# mkfs.xfs /dev/mapper/sotrage
meta-data=/dev/mapper/sotrage isize=512 agcount=4, agsize=13107200 blks
= sectsz=4096 attr=2, projid32bit=1
= crc=1 finobt=1, sparse=1, rmapbt=0
= reflink=1
data = bsize=4096 blocks=52428800, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0, ftype=1
log =internal log bsize=4096 blocks=25600, version=2
= sectsz=4096 sunit=1 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
[root@localhost ~]# udevadm settle
[root@localhost ~]# mkdir /sotrage
[root@localhost ~]# mount /dev/mapper/sotrage /sotrage/
5, View device usage
Using the command vdostats, the function of the human readable parameter is to automatically carry the storage capacity and output it in a more readable way (for example, display 20G instead of 20971520K)
[root@localhost ~]# vdostats --human-readable Device Size Used Available Use% Space saving% /dev/mapper/sotrage 20.0G 4.0G 16.0G 20% 99% [root@localhost ~]# df -h Filesystem Size Used Avail Use% Mounted on devtmpfs 889M 0 889M 0% /dev tmpfs 904M 0 904M 0% /dev/shm tmpfs 904M 9.7M 894M 2% /run tmpfs 904M 0 904M 0% /sys/fs/cgroup /dev/mapper/rhel-root 17G 3.9G 14G 23% / /dev/nvme0n1p1 1014M 169M 846M 17% /boot tmpfs 181M 16K 181M 1% /run/user/42 tmpfs 181M 2.3M 179M 2% /run/user/1000 /dev/sr0 6.7G 6.7G 0 100% /media/cdrom tmpfs 181M 4.0K 181M 1% /run/user/0 /dev/mapper/sotrage 200G 1.5G 199G 1% /sotrage
6, Storage file test
[root@localhost ~]# ls -lh /media/cdrom/images/install.img -r--r--r--. 1 roo roo 448M Apr 4 2019 /media/cdrom/images/install.img [root@localhost ~]# cp /media/cdrom/images/install.img /sotrage/ [root@localhost ~]# ls -lh /sotrage/install.img -r--r--r--. 1 root root 448M Oct 20 00:45 /sotrage/install.img [root@localhost ~]# vdostats --human-readable Device Size Used Available Use% Space saving% /dev/mapper/sotrage 20.0G 4.4G 15.6G 22% 18% [root@localhost ~]# cp /media/cdrom/images/install.img /sotrage/ cp: overwrite '/sotrage/install.img'? ^C [root@localhost ~]# cp /media/cdrom/images/install.img /sotrage/ss.img [root@localhost ~]# vdostats --human-readable Device Size Used Available Use% Space saving% /dev/mapper/sotrage 20.0G 4.5G 15.5G 22% 55%
You can see that the second copy of the same files in the past only takes up 100M of space
7, Permanently mounted for use
[root@localhost ~]# blkid /dev/mapper/sotrage /dev/mapper/sotrage: UUID="e1b143ff-e663-4ca8-8e82-959430835086" TYPE="xfs" [root@localhost ~]# vi /etc/fstab # /etc/fstab # Created by anaconda on Wed Mar 17 00:16:10 2021 # # Accessible filesystems, by reference, are maintained under '/dev/disk/'. # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. # # After editing this file, run 'systemctl daemon-reload' to update systemd # units generated from this file. # /dev/mapper/rhel-root / xfs defaults 0 0 UUID=72e9e1b7-3326-413e-956f-d2ee7e41ad4a /boot xfs defaults 0 0 /dev/mapper/rhel-swap swap swap defaults 0 0 UUID="e1b143ff-e663-4ca8-8e82-959430835086" /storage xfs defaults,_netdev 0 0