ISCSI存储
slave-147作为服务端
需要安装的软件
[root@slave-147 ~]# yum install -y scsi-target-utils
slave-148和slave-149作为客户端
需要安装的软件
[root@slave-148 ~]# yum install iscsi-initiator-utils
创建共享分区
[root@slave-147 ~]# fdisk /dev/sdb
Command (m for help): p
Disk /dev/sdb: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xba211077
Device Boot Start End Blocks Id System
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-2610, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-2610, default 2610): +5G
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
同样步骤再创建sdb2.
格式化为ext4文件系统：
[root@slave-147 ~]# mkfs.ext4 /dev/sdb1
[root@slave-147 ~]# mkfs.ext4 /dev/sdb2
因为我在安装系统时，磁盘创建成了LVM，可能是这个原因，导致在这里直接改/etc/tgt/targets.conf文件不生效。
所以这里我也将新建的sdb1和sdb2创建成了LVM格式：
[root@slave-147 ~]# pvcreate /dev/sdb{1,2}
Physical volume "/dev/sdb1" successfully created
Physical volume "/dev/sdb2" successfully created
[root@slave-147 ~]# pvscan
PV /dev/sdb1 lvm2 [5.01 GiB]
PV /dev/sdb2 lvm2 [5.01 GiB]
Total: 2 [10.02 GiB] / in use: 0 [0 ] / in no VG: 2 [10.02 GiB]
[root@slave-147 ~]# vgcreate vg_01 /dev/sdb{1,2}
Volume group "vg_01" successfully created
[root@slave-147 ~]# lvcreate -L 10G vg_01
Logical volume "lvol0" created
[root@slave-147 ~]# lvscan
ACTIVE '/dev/vg_01/lvol0' [10.00 GiB] inherit
[root@slave-147 ~]# lvdisplay
--- Logical volume ---
LV Path /dev/vg_01/lvol0 //地址要记住，修改targets.conf时用
LV Name lvol0
VG Name vg_01
LV UUID BAnblY-DKHO-89bT-SoRS-daCR-E76l-XeW94v
LV Write Access read/write
LV Creation host, time slave-147, 2018-03-29 13:38:44 +0800
LV Status available
# open 0
LV Size 10.00 GiB
Current LE 2560
Segments 2
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:0
[root@slave-147 ~]# vim /etc/tgt/targets.conf
#<target iqn.2008-09.com.example:server.target4> //在此段下面添加如下内容
# direct-store /dev/sdb # Becomes LUN 1
# direct-store /dev/sdc # Becomes LUN 2
# direct-store /dev/sdd # Becomes LUN 3
# write-cache off
# vendor_id MyCompany Inc.
#</target>
<target iqn.2018-01.cn.xueji.www:target_san1>
backing-store /dev/vg_01/lvol0
initiator-address 192.168.75.148
vendor_id xueji
product_id target1
</target>
重启tgtd服务
[root@slave-147 ~]# /etc/init.d/tgtd restart
查看：
[root@slave-147 ~]# tgt-admin --show
Target 1: iqn.2018-01.cn.xueji.www:target_san1
System information:
Driver: iscsi
State: ready
I_T nexus information:
LUN information:
LUN: 0
Type: controller
SCSI ID: IET 00010000
SCSI SN: beaf10
Size: 0 MB, Block size: 1
Online: Yes
Removable media: No
Prevent removal: No
Readonly: No
Backing store type: null
Backing store path: None
Backing store flags:
LUN: 1
Type: disk
SCSI ID: IET 00010001
SCSI SN: beaf11
Size: 10737 MB, Block size: 512
Online: Yes
Removable media: No
Prevent removal: No
Readonly: No
Backing store type: rdwr
Backing store path: /dev/vg_01/lvol0
Backing store flags:
Account information:
ACL information:
192.168.75.148
客户端：
安装ISCSI
[root@slave-148 ~]# yum install -y iscsi-initiator-utils
[root@slave-148 ~]# /etc/init.d/iscsi start //没有任何输出，这里需要我们发现target存储，再启动客户端服务，才会生效
[root@slave-148 ~]# iscsiadm -m discovery -t sendtargets -p 192.168.75.147:3260
Starting iscsid: [ OK ] [ OK ]
192.168.75.147:3260,1 iqn.2018-01.cn.xueji.www:target_san1
[root@slave-148 ~]# /etc/init.d/iscsid status
iscsid (pid 2801) is running...
[root@slave-148 ~]# yum install -y tree //用于查询目录树
[root@slave-148 ~]# tree /var/lib/iscsi/ //发现target后，会在该目录下生成树结构
/var/lib/iscsi/
├── ifaces
├── isns
├── nodes
│ └── iqn.2018-01.cn.xueji.www:target_san1
│ └── 192.168.75.147,3260,1
│ └── default
├── send_targets
│ └── 192.168.75.147,3260
│ ├── iqn.2018-01.cn.xueji.www:target_san1,192.168.75.147,3260,1,default -> /var/lib/iscsi/nodes/iqn.2018-01.cn.xueji.www:target_san1/192.168.75.147,3260,1
│ └── st_config
├── slp
└── static
10 directories, 2 files
重启服务，查看是否发现硬盘
[root@slave-148 ~]# /etc/init.d/iscsid restart
Stopping iscsid:
Starting iscsid: [ OK ]
[root@slave-148 ~]# /etc/init.d/iscsi restart
Stopping iscsi: [ OK ]
Starting iscsi: [ OK ]
[root@slave-148 ~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 20G 0 disk
├─sda1 8:1 0 200M 0 part /boot
├─sda2 8:2 0 4G 0 part [SWAP]
└─sda3 8:3 0 15.8G 0 part /
sr0 11:0 1 3.6G 0 rom /mnt/cdrom
sdb 8:16 0 10G 0 disk
使用ISCSI
[root@slave-148 ~]# fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x48b9f524.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
switch off the mode (command 'c') and change display units to
sectors (command 'u').
Command (m for help): p
Disk /dev/sdb: 10.7 GB, 10737418240 bytes
64 heads, 32 sectors/track, 10240 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x48b9f524
Device Boot Start End Blocks Id System
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-10240, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-10240, default 10240): +1G
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
[root@slave-148 ~]# mkfs.ext4 /dev/sdb1
[root@slave-148 ~]# mkdir /sdb1
[root@slave-148 ~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 20G 0 disk
├─sda1 8:1 0 200M 0 part /boot
├─sda2 8:2 0 4G 0 part [SWAP]
└─sda3 8:3 0 15.8G 0 part /
sr0 11:0 1 3.6G 0 rom /mnt/cdrom
sdb 8:16 0 10G 0 disk
└─sdb1 8:17 0 1G 0 part
[root@slave-148 ~]# mount /dev/sb1 /sdb1/
[root@slave-148 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda3 16G 4.1G 11G 28% /
tmpfs 996M 72K 996M 1% /dev/shm
/dev/sda1 194M 34M 151M 19% /boot
/dev/sr0 3.6G 3.6G 0 100% /mnt/cdrom
[root@slave-148 sdb1]# touch slave-148.txt
[root@slave-148 sdb1]# ll
total 16
drwx------ 2 root root 16384 Mar 29 14:22 lost+found
-rw-r--r-- 1 root root 0 Mar 29 14:25 slave-148.txt
为了验证，我们再以同样的方式添加一台客户端
首先将slave-148上的iscsi停掉，这里要注意，必须先停掉，再去添加，否则会出错。具体原因还有待研究。
[root@slave-148 ~]# /etc/init.d/iscsi stop
Stopping iscsi: [ OK ]
在master-147上：
<target iqn.2018-01.cn.xueji.www:target_san1>
backing-store /dev/vg_01/lvol0
initiator-address 192.168.75.148
initiator-address 192.168.75.149 //在原有的基础上加上这行
vendor_id xueji
product_id target1
</target>
[root@slave-147 ~]# /etc/init.d/tgtd restart
Stopping SCSI target daemon: [ OK ]
Starting SCSI target daemon: [ OK ]
[root@slave-147 ~]# tgt-admin --show
Target 1: iqn.2018-01.cn.xueji.www:target_san1
System information:
Driver: iscsi
State: ready
I_T nexus information:
LUN information:
LUN: 0
Type: controller
SCSI ID: IET 00010000
SCSI SN: beaf10
Size: 0 MB, Block size: 1
Online: Yes
Removable media: No
Prevent removal: No
Readonly: No
Backing store type: null
Backing store path: None
Backing store flags:
LUN: 1
Type: disk
SCSI ID: IET 00010001
SCSI SN: beaf11
Size: 10737 MB, Block size: 512
Online: Yes
Removable media: No
Prevent removal: No
Readonly: No
Backing store type: rdwr
Backing store path: /dev/vg_01/lvol0
Backing store flags:
Account information:
ACL information:
192.168.75.148
192.168.75.149
在slave-149上
[root@slave-149 ~]# /etc/init.d/iscsi start
[root@slave-149 ~]# /etc/init.d/iscsid start
[root@slave-149 ~]# iscsiadm -m discovery -t sendtargets -p 192.168.75.147:3260Starting iscsid: [ OK ]
192.168.75.147:3260,1 iqn.2018-01.cn.xueji.www:target_san1
[root@slave-149 ~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 20G 0 disk
├─sda1 8:1 0 200M 0 part /boot
├─sda2 8:2 0 4G 0 part [SWAP]
└─sda3 8:3 0 15.8G 0 part /
sr0 11:0 1 3.6G 0 rom /mnt/cdrom
sdb 8:16 0 10G 0 disk
└─sdb1 8:17 0 1G 0 part
使用
[root@slave-149 ~]# mkdir /sdb1
[root@slave-149 ~]# fdisk /dev/sdb
WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
switch off the mode (command 'c') and change display units to
sectors (command 'u').
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (1026-10240, default 1026):
Using default value 1026
Last cylinder, +cylinders or +size{K,M,G} (1026-10240, default 10240): +1G
Command (m for help): p
Disk /dev/sdb: 10.7 GB, 10737418240 bytes
64 heads, 32 sectors/track, 10240 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x48b9f524
Device Boot Start End Blocks Id System
/dev/sdb1 1 1025 1049584 83 Linux
/dev/sdb2 1026 2050 1049600 83 Linux
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
[root@slave-149 ~]# mkfs.ext4 /dev/sdb2
[root@slave-149 ~]# mount /dev/sdb2 /sdb2
[root@slave-149 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda3 16G 3.9G 11G 26% /
tmpfs 996M 72K 996M 1% /dev/shm
/dev/sda1 194M 34M 151M 19% /boot
/dev/sr0 3.6G 3.6G 0 100% /mnt/cdrom
/dev/sdb1 1008M 34M 924M 4% /sdb1
/dev/sdb2 1008M 34M 924M 4% /sdb2
进入到sdb1中查看
[root@slave-149 sdb1]# ll
total 16
drwx------ 2 root root 16384 Mar 29 14:57 lost+found
-rw-r--r-- 1 root root 0 Mar 29 15:02 slave-148.txt
在slave-148上面查看
[root@slave-148 ~]# iscsiadm -m discovery -t sendtargets -p 192.168.75.147:3260192.168.75.147:3260,1 iqn.2018-03.cn.xueji.www:target_san1
[root@slave-148 ~]# /etc/init.d/iscsid restart
Stopping iscsid:
Starting iscsid: [ OK ]
[root@slave-148 ~]# /etc/init.d/iscsi restart
Stopping iscsi: [ OK ]
Starting iscsi: [ OK ]
[root@slave-148 ~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 20G 0 disk
├─sda1 8:1 0 200M 0 part /boot
├─sda2 8:2 0 4G 0 part [SWAP]
└─sda3 8:3 0 15.8G 0 part /
sr0 11:0 1 3.6G 0 rom /mnt/cdrom
sdb 8:16 0 10G 0 disk
├─sdb1 8:17 0 1G 0 part
└─sdb2 8:18 0 1G 0 part
到sdb1中查看
[root@slave-148 sdb1]# ll
total 16
drwx------ 2 root root 16384 Mar 29 14:57 lost+found
-rw-r--r-- 1 root root 0 Mar 29 15:02 slave-148.txt
-rw-r--r-- 1 root root 0 Mar 29 15:03 slave-149.tx
同步这块，还不是很清楚，在slave-148上创建了测试数据后，不能立刻在slave-149上查到相关数据，要重新发现，重启slave-149上的服务才可以，这点可能是我做的有问题，后面在折腾下。今天先记录到这里~
