使用mdadm工具实现软RAID 5实战案例
使用mdadm工具实现软RAID 5实战案例
作者：尹正杰
版权声明：原创作品，谢绝转载！否则将追究法律责任。
一.准备工作
1>.实验环境说明
本次试验我打算采用4个分区设备来做RAID 5，如下图所示。
一般情况下使用3块设备就可以做RAID 5,但我之所以使用4块设备，是想用第4块设备充当空闲硬盘，当RAID 5中某一块设备损坏时，它会立即顶上去。 温馨提示： 由于我们使用的是分区来模拟RAID实战(即软RAID)，因此在分区时务必指定Id类型为"fd"(即"Linux raid auto") 生产环境中我们很少用磁盘分区来做RAID，而是直接使用整块磁盘来做RAID阵列，而且会有专门的磁盘阵列卡。
2>.为/dev/sdb磁盘新增一个200G的分区
[root@yinzhengjie.com ~]# fdisk /dev/sdb Welcome to fdisk (util-linux 2.23.2). Changes will remain in memory only, until you decide to write them. Be careful before using the write command. Command (m for help): p Disk /dev/sdb: 644.2 GB, 644245094400 bytes, 1258291200 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: dos Disk identifier: 0x8c3b1f6e Device Boot Start End Blocks Id System /dev/sdb1 2048 209717247 104857600 fd Linux raid autodetect Command (m for help): n Partition type: p primary (1 primary, 0 extended, 3 free) e extended Select (default p): p Partition number (2-4, default 2): First sector (209717248-1258291199, default 209717248): Using default value 209717248 Last sector, +sectors or +size{K,M,G} (209717248-1258291199, default 1258291199): +200G Partition 2 of type Linux and of size 200 GiB is set Command (m for help): p Disk /dev/sdb: 644.2 GB, 644245094400 bytes, 1258291200 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: dos Disk identifier: 0x8c3b1f6e Device Boot Start End Blocks Id System /dev/sdb1 2048 209717247 104857600 fd Linux raid autodetect /dev/sdb2 209717248 629147647 209715200 83 Linux Command (m for help): t Partition number (1,2, default 2): 2 Hex code (type L to list all codes): L 0 Empty 24 NEC DOS 81 Minix / old Lin bf Solaris 1 FAT12 27 Hidden NTFS Win 82 Linux swap / So c1 DRDOS/sec (FAT- 2 XENIX root 39 Plan 9 83 Linux c4 DRDOS/sec (FAT- 3 XENIX usr 3c PartitionMagic 84 OS/2 hidden C: c6 DRDOS/sec (FAT- 4 FAT16 <32M 40 Venix 80286 85 Linux extended c7 Syrinx 5 Extended 41 PPC PReP Boot 86 NTFS volume set da Non-FS data 6 FAT16 42 SFS 87 NTFS volume set db CP/M / CTOS / . 7 HPFS/NTFS/exFAT 4d QNX4.x 88 Linux plaintext de Dell Utility 8 AIX 4e QNX4.x 2nd part 8e Linux LVM df BootIt 9 AIX bootable 4f QNX4.x 3rd part 93 Amoeba e1 DOS access a OS/2 Boot Manag 50 OnTrack DM 94 Amoeba BBT e3 DOS R/O b W95 FAT32 51 OnTrack DM6 Aux 9f BSD/OS e4 SpeedStor c W95 FAT32 (LBA) 52 CP/M a0 IBM Thinkpad hi eb BeOS fs e W95 FAT16 (LBA) 53 OnTrack DM6 Aux a5 FreeBSD ee GPT f W95 Ext'd (LBA) 54 OnTrackDM6 a6 OpenBSD ef EFI (FAT-12/16/ 10 OPUS 55 EZ-Drive a7 NeXTSTEP f0 Linux/PA-RISC b 11 Hidden FAT12 56 Golden Bow a8 Darwin UFS f1 SpeedStor 12 Compaq diagnost 5c Priam Edisk a9 NetBSD f4 SpeedStor 14 Hidden FAT16 <3 61 SpeedStor ab Darwin boot f2 DOS secondary 16 Hidden FAT16 63 GNU HURD or Sys af HFS / HFS+ fb VMware VMFS 17 Hidden HPFS/NTF 64 Novell Netware b7 BSDI fs fc VMware VMKCORE 18 AST SmartSleep 65 Novell Netware b8 BSDI swap fd Linux raid auto 1b Hidden W95 FAT3 70 DiskSecure Mult bb Boot Wizard hid fe LANstep 1c Hidden W95 FAT3 75 PC/IX be Solaris boot ff BBT 1e Hidden W95 FAT1 80 Old Minix Hex code (type L to list all codes): fd Changed type of partition 'Linux' to 'Linux raid autodetect' Command (m for help): p Disk /dev/sdb: 644.2 GB, 644245094400 bytes, 1258291200 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: dos Disk identifier: 0x8c3b1f6e Device Boot Start End Blocks Id System /dev/sdb1 2048 209717247 104857600 fd Linux raid autodetect /dev/sdb2 209717248 629147647 209715200 fd Linux raid autodetect Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. WARNING: Re-reading the partition table failed with error 16: Device or resource busy. The kernel still uses the old table. The new table will be used at the next reboot or after you run partprobe(8) or kpartx(8) Syncing disks. [root@yinzhengjie.com ~]#
3>.为/dev/sdc磁盘新增一个200G的分区
[root@yinzhengjie.com ~]# fdisk /dev/sdc Welcome to fdisk (util-linux 2.23.2). Changes will remain in memory only, until you decide to write them. Be careful before using the write command. Command (m for help): p Disk /dev/sdc: 429.5 GB, 429496729600 bytes, 838860800 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: dos Disk identifier: 0x8c3b1f6e Device Boot Start End Blocks Id System /dev/sdc1 2048 209717247 104857600 fd Linux raid autodetect Command (m for help): n Partition type: p primary (1 primary, 0 extended, 3 free) e extended Select (default p): Using default response p Partition number (2-4, default 2): First sector (209717248-838860799, default 209717248): Using default value 209717248 Last sector, +sectors or +size{K,M,G} (209717248-838860799, default 838860799): +200G Partition 2 of type Linux and of size 200 GiB is set Command (m for help): t Partition number (1,2, default 2): Hex code (type L to list all codes): fd Changed type of partition 'Linux' to 'Linux raid autodetect' Command (m for help): p Disk /dev/sdc: 429.5 GB, 429496729600 bytes, 838860800 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: dos Disk identifier: 0x8c3b1f6e Device Boot Start End Blocks Id System /dev/sdc1 2048 209717247 104857600 fd Linux raid autodetect /dev/sdc2 209717248 629147647 209715200 fd Linux raid autodetect Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. WARNING: Re-reading the partition table failed with error 16: Device or resource busy. The kernel still uses the old table. The new table will be used at the next reboot or after you run partprobe(8) or kpartx(8) Syncing disks. [root@yinzhengjie.com ~]#
4>.为/dev/sdd磁盘新增一个200G的分区
[root@yinzhengjie.com ~]# fdisk /dev/sdd Welcome to fdisk (util-linux 2.23.2). Changes will remain in memory only, until you decide to write them. Be careful before using the write command. Device does not contain a recognized partition table Building a new DOS disklabel with disk identifier 0xed751bb5. Command (m for help): p Disk /dev/sdd: 859.0 GB, 858993459200 bytes, 1677721600 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: dos Disk identifier: 0xed751bb5 Device Boot Start End Blocks Id System Command (m for help): n Partition type: p primary (0 primary, 0 extended, 4 free) e extended Select (default p): Using default response p Partition number (1-4, default 1): First sector (2048-1677721599, default 2048): Using default value 2048 Last sector, +sectors or +size{K,M,G} (2048-1677721599, default 1677721599): +200G Partition 1 of type Linux and of size 200 GiB is set Command (m for help): t Selected partition 1 Hex code (type L to list all codes): fd Changed type of partition 'Linux' to 'Linux raid autodetect' Command (m for help): p Disk /dev/sdd: 859.0 GB, 858993459200 bytes, 1677721600 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: dos Disk identifier: 0xed751bb5 Device Boot Start End Blocks Id System /dev/sdd1 2048 419432447 209715200 fd Linux raid autodetect Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks. [root@yinzhengjie.com ~]#
5>.为/dev/sde磁盘新增一个200G的分区
[root@yinzhengjie.com ~]# fdisk /dev/sde Welcome to fdisk (util-linux 2.23.2). Changes will remain in memory only, until you decide to write them. Be careful before using the write command. Device does not contain a recognized partition table Building a new DOS disklabel with disk identifier 0x487f53ac. Command (m for help): p Disk /dev/sde: 1073.7 GB, 1073741824000 bytes, 2097152000 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: dos Disk identifier: 0x487f53ac Device Boot Start End Blocks Id System Command (m for help): n Partition type: p primary (0 primary, 0 extended, 4 free) e extended Select (default p): Using default response p Partition number (1-4, default 1): First sector (2048-2097151999, default 2048): Using default value 2048 Last sector, +sectors or +size{K,M,G} (2048-2097151999, default 2097151999): +200G Partition 1 of type Linux and of size 200 GiB is set Command (m for help): P Disk /dev/sde: 1073.7 GB, 1073741824000 bytes, 2097152000 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: dos Disk identifier: 0x487f53ac Device Boot Start End Blocks Id System /dev/sde1 2048 419432447 209715200 83 Linux Command (m for help): t Selected partition 1 Hex code (type L to list all codes): fd Changed type of partition 'Linux' to 'Linux raid autodetect' Command (m for help): p Disk /dev/sde: 1073.7 GB, 1073741824000 bytes, 2097152000 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: dos Disk identifier: 0x487f53ac Device Boot Start End Blocks Id System /dev/sde1 2048 419432447 209715200 fd Linux raid autodetect Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks. [root@yinzhengjie.com ~]#
二.创建RAID 5块设备实战案例
1>.创建一个RAID 5设备
[root@yinzhengjie.com ~]# cat /proc/mdstat Personalities : [raid0] md1 : active raid0 sdc1[1] sdb1[0] 209582080 blocks super 1.2 512k chunks unused devices: <none> [root@yinzhengjie.com ~]# [root@yinzhengjie.com ~]# mdadm -C -a yes /dev/md2 -l 5 -n 3 -x 1 -c 1M /dev/sd{b2,c2,d1,e1} #"-C -a yes"表示创建一个RAID设备，名称为"/dev/md2"。"-x"表示备用的设备，"-c"表示组块的大小为1M，默认最后一块设备(/dev/sde1)为备用块设备。 mdadm: /dev/sdb2 appears to contain an ext2fs file system size=209715200K mtime=Thu Jan 1 08:00:00 1970 Continue creating array? yes mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md2 started. [root@yinzhengjie.com ~]# [root@yinzhengjie.com ~]# cat /proc/mdstat #当创建RAID设备时，由于块设备较大，因此需要等待一段时间，此时我们就可以看到创建过程的进度，如下所示，可以看到当前的进度时6.2%。 Personalities : [raid0] [raid6] [raid5] [raid4] md2 : active raid5 sdd1[4] sde1[3](S) sdc2[1] sdb2[0] 419166208 blocks super 1.2 level 5, 1024k chunk, algorithm 2 [3/2] [UU_] [=>...................] recovery = 6.2% (13109760/209583104) finish=7.2min speed=452060K/sec bitmap: 0/2 pages [0KB], 65536KB chunk md1 : active raid0 sdc1[1] sdb1[0] 209582080 blocks super 1.2 512k chunks unused devices: <none> [root@yinzhengjie.com ~]# [root@yinzhengjie.com ~]# cat /proc/mdstat #目前已经创建完毕啦~之前的进度信息也没啦！ Personalities : [raid0] [raid6] [raid5] [raid4] md2 : active raid5 sdd1[4] sde1[3](S) sdc2[1] sdb2[0] 419166208 blocks super 1.2 level 5, 1024k chunk, algorithm 2 [3/3] [UUU] bitmap: 0/2 pages [0KB], 65536KB chunk md1 : active raid0 sdc1[1] sdb1[0] 209582080 blocks super 1.2 512k chunks unused devices: <none> [root@yinzhengjie.com ~]#
2>.查看创建设备的详细信息
[root@yinzhengjie.com ~]# mdadm -D /dev/md2 /dev/md2: Version : 1.2 Creation Time : Fri Aug 21 03:27:31 2020 Raid Level : raid5 Array Size : 419166208 (399.75 GiB 429.23 GB) Used Dev Size : 209583104 (199.87 GiB 214.61 GB) Raid Devices : 3 Total Devices : 4 Persistence : Superblock is persistent Intent Bitmap : Internal Update Time : Fri Aug 21 03:35:01 2020 State : clean Active Devices : 3 Working Devices : 4 Failed Devices : 0 Spare Devices : 1 Layout : left-symmetric Chunk Size : 1024K Consistency Policy : bitmap Name : yinzhengjie.com:2 (local to host yinzhengjie.com) UUID : 7bdb7eab:498a99bb:f69ee4ce:7d5c010f Events : 93 Number Major Minor RaidDevice State 0 8 18 0 active sync /dev/sdb2 1 8 34 1 active sync /dev/sdc2 4 8 49 2 active sync /dev/sdd1 3 8 65 - spare /dev/sde1 [root@yinzhengjie.com ~]# [root@yinzhengjie.com ~]#
3>.对RAID 5设备进行格式化，指定为XFS文件系统
[root@yinzhengjie.com ~]# blkid /dev/sdc1: UUID="e7e1a4a7-f73b-2990-a52b-16b392be0561" UUID_SUB="78244c8b-f961-577f-9d32-af6c474e9f2b" LABEL="yinzhengjie.com:001" TYPE="linux_raid_member" /dev/sdc2: UUID="7bdb7eab-498a-99bb-f69e-e4ce7d5c010f" UUID_SUB="142f3a98-d654-1abd-00bd-a50a66fec7fc" LABEL="yinzhengjie.com:2" TYPE="linux_raid_member" /dev/sdd1: UUID="7bdb7eab-498a-99bb-f69e-e4ce7d5c010f" UUID_SUB="f1699604-e5f0-f054-31f6-a56f90d452e3" LABEL="yinzhengjie.com:2" TYPE="linux_raid_member" /dev/sdb1: UUID="e7e1a4a7-f73b-2990-a52b-16b392be0561" UUID_SUB="595fb659-a792-aaac-3a2d-800b3e0795ac" LABEL="yinzhengjie.com:001" TYPE="linux_raid_member" /dev/sdb2: UUID="7bdb7eab-498a-99bb-f69e-e4ce7d5c010f" UUID_SUB="5a6802d5-93af-680d-dca1-64d02c5b615b" LABEL="yinzhengjie.com:2" TYPE="linux_raid_member" /dev/sde1: UUID="7bdb7eab-498a-99bb-f69e-e4ce7d5c010f" UUID_SUB="e6b567f9-0b34-3acd-6d7d-39540d51878f" LABEL="yinzhengjie.com:2" TYPE="linux_raid_member" /dev/sda1: PARTUUID="8f500d9c-87b9-421d-8ce0-8c385b73bf08" /dev/sda2: UUID="1865a93f-6113-4097-89dc-8c4ea5fdf68c" TYPE="xfs" PARTUUID="e85154fa-bbc8-43c1-90f0-676000195a97" /dev/sda3: UUID="lNmzXe-Whhd-NTtI-C55A-7m4x-CudG-1Q556q" TYPE="LVM2_member" PARTUUID="fa2e251a-d220-415f-acc7-da94b7a2d1b4" /dev/sda4: PARTUUID="814caa31-3a7e-499d-8b7d-cc3017473ba8" /dev/sr0: UUID="2018-11-25-21-21-31-00" LABEL="CentOS 7 x86_64" TYPE="iso9660" PTTYPE="dos" /dev/mapper/centos-root: UUID="c0160ecb-06e4-497e-813c-53690b4a6109" TYPE="xfs" /dev/md1: UUID="bc7d4ffa-439e-4544-b59e-58d10228569c" TYPE="xfs" /dev/mapper/centos-yinzhengjie: UUID="6a2643fb-8d55-44cd-b41e-b5f2709ef855" TYPE="xfs" [root@yinzhengjie.com ~]# [root@yinzhengjie.com ~]# mkfs.xfs /dev/md2 #格式化RAID 5设备，将其格式化为XFS文件系统。 meta-data=/dev/md2 isize=512 agcount=16, agsize=6549248 blks = sectsz=512 attr=2, projid32bit=1 = crc=1 finobt=0, sparse=0 data = bsize=4096 blocks=104787968, imaxpct=25 = sunit=256 swidth=512 blks naming =version 2 bsize=4096 ascii-ci=0 ftype=1 log =internal log bsize=4096 blocks=51168, version=2 = sectsz=512 sunit=8 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 [root@yinzhengjie.com ~]# [root@yinzhengjie.com ~]# blkid /dev/sdc1: UUID="e7e1a4a7-f73b-2990-a52b-16b392be0561" UUID_SUB="78244c8b-f961-577f-9d32-af6c474e9f2b" LABEL="yinzhengjie.com:001" TYPE="linux_raid_member" /dev/sdc2: UUID="7bdb7eab-498a-99bb-f69e-e4ce7d5c010f" UUID_SUB="142f3a98-d654-1abd-00bd-a50a66fec7fc" LABEL="yinzhengjie.com:2" TYPE="linux_raid_member" /dev/sdd1: UUID="7bdb7eab-498a-99bb-f69e-e4ce7d5c010f" UUID_SUB="f1699604-e5f0-f054-31f6-a56f90d452e3" LABEL="yinzhengjie.com:2" TYPE="linux_raid_member" /dev/sdb1: UUID="e7e1a4a7-f73b-2990-a52b-16b392be0561" UUID_SUB="595fb659-a792-aaac-3a2d-800b3e0795ac" LABEL="yinzhengjie.com:001" TYPE="linux_raid_member" /dev/sdb2: UUID="7bdb7eab-498a-99bb-f69e-e4ce7d5c010f" UUID_SUB="5a6802d5-93af-680d-dca1-64d02c5b615b" LABEL="yinzhengjie.com:2" TYPE="linux_raid_member" /dev/sde1: UUID="7bdb7eab-498a-99bb-f69e-e4ce7d5c010f" UUID_SUB="e6b567f9-0b34-3acd-6d7d-39540d51878f" LABEL="yinzhengjie.com:2" TYPE="linux_raid_member" /dev/sda2: UUID="1865a93f-6113-4097-89dc-8c4ea5fdf68c" TYPE="xfs" PARTUUID="e85154fa-bbc8-43c1-90f0-676000195a97" /dev/sda3: UUID="lNmzXe-Whhd-NTtI-C55A-7m4x-CudG-1Q556q" TYPE="LVM2_member" PARTUUID="fa2e251a-d220-415f-acc7-da94b7a2d1b4" /dev/sr0: UUID="2018-11-25-21-21-31-00" LABEL="CentOS 7 x86_64" TYPE="iso9660" PTTYPE="dos" /dev/mapper/centos-root: UUID="c0160ecb-06e4-497e-813c-53690b4a6109" TYPE="xfs" /dev/md1: UUID="bc7d4ffa-439e-4544-b59e-58d10228569c" TYPE="xfs" /dev/mapper/centos-yinzhengjie: UUID="6a2643fb-8d55-44cd-b41e-b5f2709ef855" TYPE="xfs" /dev/sda1: PARTUUID="8f500d9c-87b9-421d-8ce0-8c385b73bf08" /dev/sda4: PARTUUID="814caa31-3a7e-499d-8b7d-cc3017473ba8" /dev/md2: UUID="eb94af80-9d27-4c8a-9c7b-1f54bb40048f" TYPE="xfs" [root@yinzhengjie.com ~]#
4>.挂载设备
[root@yinzhengjie.com ~]# vim /etc/fstab [root@yinzhengjie.com ~]# [root@yinzhengjie.com ~]# mkdir /yinzhengjie/raid5 [root@yinzhengjie.com ~]# [root@yinzhengjie.com ~]# tail -1 /etc/fstab UUID=eb94af80-9d27-4c8a-9c7b-1f54bb40048f /yinzhengjie/raid5 xfs defaults,noatime,nodiratime 0 0 [root@yinzhengjie.com ~]# [root@yinzhengjie.com ~]# mount -a [root@yinzhengjie.com ~]# [root@yinzhengjie.com ~]# df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/centos-root 500G 14G 486G 3% / devtmpfs 2.0G 0 2.0G 0% /dev tmpfs 2.0G 0 2.0G 0% /dev/shm tmpfs 2.0G 12M 2.0G 1% /run tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup /dev/sda2 1014M 133M 882M 14% /boot /dev/mapper/centos-yinzhengjie 1.6T 2.4G 1.6T 1% /yinzhengjie /dev/md1 200G 1.1G 199G 1% /yinzhengjie/raid tmpfs 394M 0 394M 0% /run/user/0 /dev/md2 400G 33M 400G 1% /yinzhengjie/raid5 #为什么4块200GB的设备，但其大小仅有400G呢？ [root@yinzhengjie.com ~]#
5>.测试RAID 5设备的I/O性能
[root@yinzhengjie.com ~]# free -h total used free shared buff/cache available Mem: 3.8G 259M 3.4G 11M 143M 3.4G Swap: 0B 0B 0B [root@yinzhengjie.com ~]# [root@yinzhengjie.com ~]# dd if=/dev/zero of=/yinzhengjie/raid5/test.log bs=1M count=1024 #测试RAID 5设备的写性能 1024+0 records in 1024+0 records out 1073741824 bytes (1.1 GB) copied, 1.86494 s, 576 MB/s [root@yinzhengjie.com ~]# [root@yinzhengjie.com ~]# free -h total used free shared buff/cache available Mem: 3.8G 263M 2.4G 11M 1.2G 3.3G Swap: 0B 0B 0B [root@yinzhengjie.com ~]# [root@yinzhengjie.com ~]# sync; echo 1 > /proc/sys/vm/drop_caches [root@yinzhengjie.com ~]# sync; echo 2 > /proc/sys/vm/drop_caches [root@yinzhengjie.com ~]# sync; echo 3 > /proc/sys/vm/drop_caches [root@yinzhengjie.com ~]# [root@yinzhengjie.com ~]# free -h total used free shared buff/cache available Mem: 3.8G 261M 3.4G 11M 143M 3.4G Swap: 0B 0B 0B [root@yinzhengjie.com ~]# [root@yinzhengjie.com ~]# dd if=/yinzhengjie/raid5/test.log of=/dev/null #测试RAID 5设备的读性能 2097152+0 records in 2097152+0 records out 1073741824 bytes (1.1 GB) copied, 2.85041 s, 377 MB/s [root@yinzhengjie.com ~]#
三.测试RAID 5设备
1>.备份mdadm的配置信息
[root@yinzhengjie.com ~]# mdadm -Ds ARRAY /dev/md/001 metadata=1.2 name=yinzhengjie.com:001 UUID=e7e1a4a7:f73b2990:a52b16b3:92be0561 ARRAY /dev/md2 metadata=1.2 spares=1 name=yinzhengjie.com:2 UUID=7bdb7eab:498a99bb:f69ee4ce:7d5c010f [root@yinzhengjie.com ~]# [root@yinzhengjie.com ~]# mdadm -Ds >/etc/mdadm.conf #将mdadm管理的RAID设备信息导出到一个配置文件中，如果人为损坏设备恢复时可以用到对应的UUID。 [root@yinzhengjie.com ~]# [root@yinzhengjie.com ~]# cat /etc/mdadm.conf ARRAY /dev/md/001 metadata=1.2 name=yinzhengjie.com:001 UUID=e7e1a4a7:f73b2990:a52b16b3:92be0561 ARRAY /dev/md2 metadata=1.2 spares=1 name=yinzhengjie.com:2 UUID=7bdb7eab:498a99bb:f69ee4ce:7d5c010f [root@yinzhengjie.com ~]#
2>.手动将RAID 5中的一块设备下线，观察备用设备是否会自动构建
[root@yinzhengjie.com ~]# mdadm -D /dev/md2 #观察操作之前的RAID 5设备信息 /dev/md2: Version : 1.2 Creation Time : Fri Aug 21 03:27:31 2020 Raid Level : raid5 Array Size : 419166208 (399.75 GiB 429.23 GB) Used Dev Size : 209583104 (199.87 GiB 214.61 GB) Raid Devices : 3 Total Devices : 4 Persistence : Superblock is persistent Intent Bitmap : Internal Update Time : Fri Aug 21 05:40:34 2020 State : clean Active Devices : 3 Working Devices : 4 Failed Devices : 0 Spare Devices : 1 Layout : left-symmetric Chunk Size : 1024K Consistency Policy : bitmap Name : yinzhengjie.com:2 (local to host yinzhengjie.com) UUID : 7bdb7eab:498a99bb:f69ee4ce:7d5c010f Events : 93 Number Major Minor RaidDevice State 0 8 18 0 active sync /dev/sdb2 1 8 34 1 active sync /dev/sdc2 4 8 49 2 active sync /dev/sdd1 3 8 65 - spare /dev/sde1 [root@yinzhengjie.com ~]#
[root@yinzhengjie.com ~]# mdadm /dev/md2 -f /dev/sdc2 #手动下线"/dev/sdc2"这块设备，请仔细观察"RAID 5"设备的信息，你会发现空闲设备自动就顶替上来啦，如下图所示。 mdadm: set /dev/sdc2 faulty in /dev/md2 [root@yinzhengjie.com ~]# [root@yinzhengjie.com ~]# mdadm -D /dev/md2 /dev/md2: Version : 1.2 Creation Time : Fri Aug 21 03:27:31 2020 Raid Level : raid5 Array Size : 419166208 (399.75 GiB 429.23 GB) Used Dev Size : 209583104 (199.87 GiB 214.61 GB) Raid Devices : 3 Total Devices : 4 Persistence : Superblock is persistent Intent Bitmap : Internal Update Time : Fri Aug 21 05:51:07 2020 State : clean, degraded, recovering Active Devices : 2 Working Devices : 3 Failed Devices : 1 Spare Devices : 1 Layout : left-symmetric Chunk Size : 1024K Consistency Policy : bitmap Rebuild Status : 0% complete Name : yinzhengjie.com:2 (local to host yinzhengjie.com) UUID : 7bdb7eab:498a99bb:f69ee4ce:7d5c010f Events : 95 Number Major Minor RaidDevice State 0 8 18 0 active sync /dev/sdb2 3 8 65 1 spare rebuilding /dev/sde1 4 8 49 2 active sync /dev/sdd1 1 8 34 - faulty /dev/sdc2 [root@yinzhengjie.com ~]#
[root@yinzhengjie.com ~]# mdadm -D /dev/md2 #当备用(Spare)设备替换了"faulty"状态的设备后，其状态"active"，中间状态为"spare rebuilding"(我测试环境的200G磁盘，需要5分钟左右才完成) /dev/md2: Version : 1.2 Creation Time : Fri Aug 21 03:27:31 2020 Raid Level : raid5 Array Size : 419166208 (399.75 GiB 429.23 GB) Used Dev Size : 209583104 (199.87 GiB 214.61 GB) Raid Devices : 3 Total Devices : 4 Persistence : Superblock is persistent Intent Bitmap : Internal Update Time : Fri Aug 21 05:59:00 2020 State : clean Active Devices : 3 Working Devices : 3 Failed Devices : 1 Spare Devices : 0 Layout : left-symmetric Chunk Size : 1024K Consistency Policy : bitmap Name : yinzhengjie.com:2 (local to host yinzhengjie.com) UUID : 7bdb7eab:498a99bb:f69ee4ce:7d5c010f Events : 191 Number Major Minor RaidDevice State 0 8 18 0 active sync /dev/sdb2 3 8 65 1 active sync /dev/sde1 4 8 49 2 active sync /dev/sdd1 1 8 34 - faulty /dev/sdc2 [root@yinzhengjie.com ~]#
3>.删除有错误的(faulty)设备
[root@yinzhengjie.com ~]# mdadm -D /dev/md2 /dev/md2: Version : 1.2 Creation Time : Fri Aug 21 03:27:31 2020 Raid Level : raid5 Array Size : 419166208 (399.75 GiB 429.23 GB) Used Dev Size : 209583104 (199.87 GiB 214.61 GB) Raid Devices : 3 Total Devices : 4 Persistence : Superblock is persistent Intent Bitmap : Internal Update Time : Fri Aug 21 05:59:00 2020 State : clean Active Devices : 3 Working Devices : 3 Failed Devices : 1 Spare Devices : 0 Layout : left-symmetric Chunk Size : 1024K Consistency Policy : bitmap Name : yinzhengjie.com:2 (local to host yinzhengjie.com) UUID : 7bdb7eab:498a99bb:f69ee4ce:7d5c010f Events : 191 Number Major Minor RaidDevice State 0 8 18 0 active sync /dev/sdb2 3 8 65 1 active sync /dev/sde1 4 8 49 2 active sync /dev/sdd1 1 8 34 - faulty /dev/sdc2 [root@yinzhengjie.com ~]#
[root@yinzhengjie.com ~]# mdadm /dev/md2 -r /dev/sdc2 #删除有错误的设备 mdadm: hot removed /dev/sdc2 from /dev/md2 [root@yinzhengjie.com ~]#
[root@yinzhengjie.com ~]# mdadm -D /dev/md2 /dev/md2: Version : 1.2 Creation Time : Fri Aug 21 03:27:31 2020 Raid Level : raid5 Array Size : 419166208 (399.75 GiB 429.23 GB) Used Dev Size : 209583104 (199.87 GiB 214.61 GB) Raid Devices : 3 Total Devices : 3 Persistence : Superblock is persistent Intent Bitmap : Internal Update Time : Fri Aug 21 06:07:15 2020 State : clean Active Devices : 3 Working Devices : 3 Failed Devices : 0 Spare Devices : 0 Layout : left-symmetric Chunk Size : 1024K Consistency Policy : bitmap Name : yinzhengjie.com:2 (local to host yinzhengjie.com) UUID : 7bdb7eab:498a99bb:f69ee4ce:7d5c010f Events : 192 Number Major Minor RaidDevice State 0 8 18 0 active sync /dev/sdb2 3 8 65 1 active sync /dev/sde1 4 8 49 2 active sync /dev/sdd1 [root@yinzhengjie.com ~]#
4>.往磁盘阵列中添加设备
[root@yinzhengjie.com ~]# mdadm -D /dev/md2 /dev/md2: Version : 1.2 Creation Time : Fri Aug 21 03:27:31 2020 Raid Level : raid5 Array Size : 419166208 (399.75 GiB 429.23 GB) Used Dev Size : 209583104 (199.87 GiB 214.61 GB) Raid Devices : 3 Total Devices : 3 Persistence : Superblock is persistent Intent Bitmap : Internal Update Time : Fri Aug 21 06:07:15 2020 State : clean Active Devices : 3 Working Devices : 3 Failed Devices : 0 Spare Devices : 0 Layout : left-symmetric Chunk Size : 1024K Consistency Policy : bitmap Name : yinzhengjie.com:2 (local to host yinzhengjie.com) UUID : 7bdb7eab:498a99bb:f69ee4ce:7d5c010f Events : 192 Number Major Minor RaidDevice State 0 8 18 0 active sync /dev/sdb2 3 8 65 1 active sync /dev/sde1 4 8 49 2 active sync /dev/sdd1 [root@yinzhengjie.com ~]#
[root@yinzhengjie.com ~]# mdadm /dev/md2 -a /dev/sdc2 #我们将设备添加到RAID 5磁盘阵列中 mdadm: re-added /dev/sdc2 [root@yinzhengjie.com ~]#
[root@yinzhengjie.com ~]# mdadm -D /dev/md2 /dev/md2: Version : 1.2 Creation Time : Fri Aug 21 03:27:31 2020 Raid Level : raid5 Array Size : 419166208 (399.75 GiB 429.23 GB) Used Dev Size : 209583104 (199.87 GiB 214.61 GB) Raid Devices : 3 Total Devices : 4 Persistence : Superblock is persistent Intent Bitmap : Internal Update Time : Fri Aug 21 06:10:38 2020 State : clean Active Devices : 3 Working Devices : 4 Failed Devices : 0 Spare Devices : 1 Layout : left-symmetric Chunk Size : 1024K Consistency Policy : bitmap Name : yinzhengjie.com:2 (local to host yinzhengjie.com) UUID : 7bdb7eab:498a99bb:f69ee4ce:7d5c010f Events : 193 Number Major Minor RaidDevice State 0 8 18 0 active sync /dev/sdb2 3 8 65 1 active sync /dev/sde1 4 8 49 2 active sync /dev/sdd1 1 8 34 - spare /dev/sdc2 [root@yinzhengjie.com ~]#
5>.验证数据是否存在
[root@yinzhengjie.com ~]# ll -h /yinzhengjie/raid5/ #当磁盘损坏时，我们发现文件依旧是可以正常访问的，原因是我们有备用的磁盘设备。 total 1.0G -rw-r--r-- 1 root root 1.0G Aug 21 05:39 test.log [root@yinzhengjie.com ~]#
6>.禁用RAID设备
[root@yinzhengjie.com ~]# df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/centos-root 500G 14G 486G 3% / devtmpfs 2.0G 0 2.0G 0% /dev tmpfs 2.0G 0 2.0G 0% /dev/shm tmpfs 2.0G 12M 2.0G 1% /run tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup /dev/sda2 1014M 133M 882M 14% /boot /dev/mapper/centos-yinzhengjie 1.6T 3.4G 1.6T 1% /yinzhengjie /dev/md1 200G 1.1G 199G 1% /yinzhengjie/raid tmpfs 394M 0 394M 0% /run/user/0 /dev/md2 400G 1.1G 399G 1% /yinzhengjie/raid5 [root@yinzhengjie.com ~]# [root@yinzhengjie.com ~]# umount /yinzhengjie/raid5/ #禁用RAID阵列之前，先要卸载对应的挂载点 [root@yinzhengjie.com ~]# [root@yinzhengjie.com ~]# df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/centos-root 500G 14G 486G 3% / devtmpfs 2.0G 0 2.0G 0% /dev tmpfs 2.0G 0 2.0G 0% /dev/shm tmpfs 2.0G 12M 2.0G 1% /run tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup /dev/sda2 1014M 133M 882M 14% /boot /dev/mapper/centos-yinzhengjie 1.6T 3.4G 1.6T 1% /yinzhengjie /dev/md1 200G 1.1G 199G 1% /yinzhengjie/raid tmpfs 394M 0 394M 0% /run/user/0 [root@yinzhengjie.com ~]# [root@yinzhengjie.com ~]#
[root@yinzhengjie.com ~]# mdadm -D /dev/md2 #查看磁盘阵列信息 /dev/md2: Version : 1.2 Creation Time : Fri Aug 21 03:27:31 2020 Raid Level : raid5 Array Size : 419166208 (399.75 GiB 429.23 GB) Used Dev Size : 209583104 (199.87 GiB 214.61 GB) Raid Devices : 3 Total Devices : 4 Persistence : Superblock is persistent Intent Bitmap : Internal Update Time : Fri Aug 21 06:25:31 2020 State : clean Active Devices : 3 Working Devices : 4 Failed Devices : 0 Spare Devices : 1 Layout : left-symmetric Chunk Size : 1024K Consistency Policy : bitmap Name : yinzhengjie.com:2 (local to host yinzhengjie.com) UUID : 7bdb7eab:498a99bb:f69ee4ce:7d5c010f Events : 193 Number Major Minor RaidDevice State 0 8 18 0 active sync /dev/sdb2 3 8 65 1 active sync /dev/sde1 4 8 49 2 active sync /dev/sdd1 1 8 34 - spare /dev/sdc2 [root@yinzhengjie.com ~]# [root@yinzhengjie.com ~]# mdadm -S /dev/md2 #禁用磁盘战列 mdadm: stopped /dev/md2 [root@yinzhengjie.com ~]# [root@yinzhengjie.com ~]# mdadm -D /dev/md2 #禁用后将无法查看磁盘阵列信息啦~ mdadm: cannot open /dev/md2: No such file or directory [root@yinzhengjie.com ~]#
7>.启用RAID设备
[root@yinzhengjie.com ~]# mdadm -D /dev/md2 #未启动磁盘阵列之前是无法查看相关信息的 mdadm: cannot open /dev/md2: No such file or directory [root@yinzhengjie.com ~]# [root@yinzhengjie.com ~]# mdadm -A /dev/md2 #启动磁盘阵列 mdadm: /dev/md2 has been started with 3 drives and 1 spare. [root@yinzhengjie.com ~]# [root@yinzhengjie.com ~]# mdadm -D /dev/md2 #启用磁盘阵列后就可以查看该阵列的相关信息啦~ /dev/md2: Version : 1.2 Creation Time : Fri Aug 21 03:27:31 2020 Raid Level : raid5 Array Size : 419166208 (399.75 GiB 429.23 GB) Used Dev Size : 209583104 (199.87 GiB 214.61 GB) Raid Devices : 3 Total Devices : 4 Persistence : Superblock is persistent Intent Bitmap : Internal Update Time : Fri Aug 21 06:25:31 2020 State : clean Active Devices : 3 Working Devices : 4 Failed Devices : 0 Spare Devices : 1 Layout : left-symmetric Chunk Size : 1024K Consistency Policy : bitmap Name : yinzhengjie.com:2 (local to host yinzhengjie.com) UUID : 7bdb7eab:498a99bb:f69ee4ce:7d5c010f Events : 193 Number Major Minor RaidDevice State 0 8 18 0 active sync /dev/sdb2 3 8 65 1 active sync /dev/sde1 4 8 49 2 active sync /dev/sdd1 1 8 34 - spare /dev/sdc2 [root@yinzhengjie.com ~]# [root@yinzhengjie.com ~]#
8>.清除磁盘的RAID信息
[root@yinzhengjie.com ~]# mdadm -D /dev/md2 /dev/md2: Version : 1.2 Creation Time : Fri Aug 21 03:27:31 2020 Raid Level : raid5 Array Size : 419166208 (399.75 GiB 429.23 GB) Used Dev Size : 209583104 (199.87 GiB 214.61 GB) Raid Devices : 3 Total Devices : 4 Persistence : Superblock is persistent Intent Bitmap : Internal Update Time : Fri Aug 21 06:25:31 2020 State : clean Active Devices : 3 Working Devices : 4 Failed Devices : 0 Spare Devices : 1 Layout : left-symmetric Chunk Size : 1024K Consistency Policy : bitmap Name : yinzhengjie.com:2 (local to host yinzhengjie.com) UUID : 7bdb7eab:498a99bb:f69ee4ce:7d5c010f Events : 193 Number Major Minor RaidDevice State 0 8 18 0 active sync /dev/sdb2 3 8 65 1 active sync /dev/sde1 4 8 49 2 active sync /dev/sdd1 1 8 34 - spare /dev/sdc2 [root@yinzhengjie.com ~]#
[root@yinzhengjie.com ~]# mdadm -S /dev/md2 #禁用磁盘阵列 mdadm: stopped /dev/md2 [root@yinzhengjie.com ~]#
[root@yinzhengjie.com ~]# blkid /dev/sd{b,c,d,e}* #查看"/dev/md2"磁盘阵列对应的分区文件，注意观察对应的UUID
/dev/sdb: PTTYPE="dos"
/dev/sdb1: UUID="e7e1a4a7-f73b-2990-a52b-16b392be0561" UUID_SUB="595fb659-a792-aaac-3a2d-800b3e0795ac" LABEL="yinzhengjie.com:001" TYPE="linux_raid_member"
/dev/sdb2: UUID="7bdb7eab-498a-99bb-f69e-e4ce7d5c010f" UUID_SUB="5a6802d5-93af-680d-dca1-64d02c5b615b" LABEL="yinzhengjie.com:2" TYPE="linux_raid_member"
/dev/sdc: PTTYPE="dos"
/dev/sdc1: UUID="e7e1a4a7-f73b-2990-a52b-16b392be0561" UUID_SUB="78244c8b-f961-577f-9d32-af6c474e9f2b" LABEL="yinzhengjie.com:001" TYPE="linux_raid_member"
/dev/sdc2: UUID="7bdb7eab-498a-99bb-f69e-e4ce7d5c010f" UUID_SUB="142f3a98-d654-1abd-00bd-a50a66fec7fc" LABEL="yinzhengjie.com:2" TYPE="linux_raid_member"
/dev/sdd: PTTYPE="dos"
/dev/sdd1: UUID="7bdb7eab-498a-99bb-f69e-e4ce7d5c010f" UUID_SUB="f1699604-e5f0-f054-31f6-a56f90d452e3" LABEL="yinzhengjie.com:2" TYPE="linux_raid_member"
/dev/sde: PTTYPE="dos"
/dev/sde1: UUID="7bdb7eab-498a-99bb-f69e-e4ce7d5c010f" UUID_SUB="e6b567f9-0b34-3acd-6d7d-39540d51878f" LABEL="yinzhengjie.com:2" TYPE="linux_raid_member"
[root@yinzhengjie.com ~]#
[root@yinzhengjie.com ~]# mdadm --zero-superblock /dev/sdb2 /dev/sdc2 /dev/sdd1 /dev/sde1 #我们将"/dev/md2"对应的RAID信息进行清空操作
[root@yinzhengjie.com ~]#
[root@yinzhengjie.com ~]# blkid /dev/sd{b,c,d,e}* #再次观察各个分区对应的UUID，发现"/dev/md2"的UUID被成功删除啦~
/dev/sdb: PTTYPE="dos"
/dev/sdb1: UUID="e7e1a4a7-f73b-2990-a52b-16b392be0561" UUID_SUB="595fb659-a792-aaac-3a2d-800b3e0795ac" LABEL="yinzhengjie.com:001" TYPE="linux_raid_member"
/dev/sdb2: LABEL="/mnt/sdb2" UUID="15028503-abd5-4bc7-9e35-2e745662d42d" TYPE="ext4"
/dev/sdc: PTTYPE="dos"
/dev/sdc1: UUID="e7e1a4a7-f73b-2990-a52b-16b392be0561" UUID_SUB="78244c8b-f961-577f-9d32-af6c474e9f2b" LABEL="yinzhengjie.com:001" TYPE="linux_raid_member"
/dev/sdc2: UUID="2307c9c5-f7b1-4a2b-b9e9-46d89a9c7c81" TYPE="xfs"
/dev/sdd: PTTYPE="dos"
/dev/sde: PTTYPE="dos"
[root@yinzhengjie.com ~]#
[root@yinzhengjie.com ~]#
本文来自博客园，作者：尹正杰，转载请注明原文链接：https://www.cnblogs.com/yinzhengjie/p/13063062.html，个人微信: "JasonYin2020"（添加时请备注来源及意图备注，有偿付费）
当你的才华还撑不起你的野心的时候，你就应该静下心来学习。当你的能力还驾驭不了你的目标的时候，你就应该沉下心来历练。问问自己，想要怎样的人生。
