Original Link: https://my.oschina.net/u/1587810/blog/266446
I encountered a performance bottleneck problem in my work and found that when the environment Guest in KVM is RHEL5.5, IO performance is much worse than RHEL6.3, so I started to study the reasons for the performance gap.Because the operating system versions are different, the focus is on the kernel and file system. Through the upgrade kernel comparison test, it is found that the performance gap is not large, but ext3 upgrade to ext4 performance significantly improved.
In view of the possible impact of qcow2 file format on performance, additional comparison experiments are performed using the raw format.The following is a comparison of test results:
RAW + ext3:
[root@localhost ~]# iozone -a -i0 -i1 -s6G -r64k Iozone: Performance Test of File I/O Version $Revision: 3.420 $ Compiled for 64 bit mode. Build: linux-ia64 Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins Al Slater, Scott Rhine, Mike Wisner, Ken Goss Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, Vangel Bojaxhi, Ben England, Vikentsi Lapa. Run began: Fri Apr 25 11:42:43 2014 Auto Mode File size set to 6291456 KB Record Size 64 KB Command line used: iozone -a -i0 -i1 -s6G -r64k Output is in Kbytes/sec Time Resolution = 0.000001 seconds. Processor cache size set to 1024 Kbytes. Processor cache line size set to 32 bytes. File stride size set to 17 * record size. random random bkwd record stride KB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread 6291456 64 422636 286223 352009 369563 iozone test complete. [root@localhost ~]# mount -l /dev/mapper/rootvg-rootlv on / type ext3 (rw) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) /dev/vda1 on /boot type ext3 (rw) [/boot1] tmpfs on /dev/shm type tmpfs (rw) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
RAW + ext4:
[root@localhost ~]# iozone -a -i0 -i1 -s6G -r64k Iozone: Performance Test of File I/O Version $Revision: 3.420 $ Compiled for 64 bit mode. Build: linux-ia64 Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins Al Slater, Scott Rhine, Mike Wisner, Ken Goss Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, Vangel Bojaxhi, Ben England, Vikentsi Lapa. Run began: Fri Apr 25 11:46:51 2014 Auto Mode File size set to 6291456 KB Record Size 64 KB Command line used: iozone -a -i0 -i1 -s6G -r64k Output is in Kbytes/sec Time Resolution = 0.000001 seconds. Processor cache size set to 1024 Kbytes. Processor cache line size set to 32 bytes. File stride size set to 17 * record size. random random bkwd record stride KB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread 6291456 64 674409 823374 443455 443929 iozone test complete. [root@localhost ~]# mount -l /dev/mapper/rootvg-rootlv on / type ext4 (rw) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) /dev/vda1 on /boot type ext4 (rw) [/boot1] tmpfs on /dev/shm type tmpfs (rw) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
The results for the qcow2 format test are as follows: (to maximize environmental fairness, CP a.qcow2 -> b.qcow2; b.qcow2 is upgraded to ext4 and then compared)
qcow2 + ext3:
Run began: Fri Apr 25 16:05:48 2014 Auto Mode File size set to 6291456 KB Record Size 64 KB Command line used: iozone -a -i0 -i1 -s6G -r64k Output is in Kbytes/sec Time Resolution = 0.000001 seconds. Processor cache size set to 1024 Kbytes. Processor cache line size set to 32 bytes. File stride size set to 17 * record size. random random bkwd record stride KB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread 6291456 64 518274 392571 262093 268514 iozone test complete.
qcow2 + ext4 :
Auto Mode File size set to 2097152 KB Record Size 64 KB Command line used: iozone -a -i0 -i1 -s2G -r64k Output is in Kbytes/sec Time Resolution = 0.000001 seconds. Processor cache size set to 1024 Kbytes. Processor cache line size set to 32 bytes. File stride size set to 17 * record size. random random bkwd record stride KB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread 2097152 64 633332 776716 312232 278960 iozone test complete. [root@localhost ~]# mount -l /dev/mapper/rootvg-rootlv on / type ext4 (rw) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) /dev/vda1 on /boot type ext4 (rw) [/boot1] tmpfs on /dev/shm type tmpfs (rw) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
Comparisons show that ext4 has significantly improved read and write performance over ext3 in any file format.
The next problem is how to upgrade rhel5.5 lossless on-line from ext3 to ext4.I took a virtual machine to do the experiment as follows:
1. Check the packages you need to be up to date:
yum install -y upgrade mkinitrd e2fsprogs yum update -y upgrade mkinitrd e2fsprogs
2. Back up fstab in case
cp /etc/fstab /etc/fstab.bak
3. Replace ext3 with ext4
sed -i "s/ext3/ext4/g" /etc/fstab
4. Back up the system initrd and rebuild the new one.Since centos5.5 does not load ext4 by default, be proactive--with=ext4
mv /boot/initrd-`uname -r`.img /boot/initrd-`uname -r`.img.bak mkinitrd -v --with=ext4 /boot/initrd-`uname -r`.img `uname -r`
5. Get the ext4 partition that does not follow the file system:
[root@localhost ~]# grep -v -w / /etc/fstab |grep ext4|awk '{print $2}' /boot /home
6. Find the corresponding device in 5:
[root@localhost ~]# df -h|egrep "boot|home" /dev/sda1 485M 67M 393M 15% /boot /dev/mapper/VolGroup-lv_home 394G 23G 352G 6% /home
umount /home
umount /boot
7. Add feature support for ext4:
tune4fs -O extents,uninit_bg,dir_index,flex_bg -m 1 /dev/mapper/VolGroup-lv_home tune4fs -O extents,uninit_bg,dir_index,flex_bg -m 1 /dev/sda1
8. Check the file system for errors and repair them automatically
fsck -vfyDC0 /dev/sda1 fsck -vfyDC0 /dev/mapper/VOlGroup-lv_home
9, Restart the system to check if ext4 is in effect
[root@localhost ~]# mount -l /dev/mapper/rootvg-rootlv on / type ext4 (rw) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) /dev/vda1 on /boot type ext4 (rw) [/boot1] tmpfs on /dev/shm type tmpfs (rw) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
Reprinted at: https://my.oschina.net/u/1587810/blog/266446