[转]iSCSI for FreeBSD

Suppose you want to use a remote iSCSI device, but you don't exactly trust either the

storage or the network in between. Of course, there's a way around it :)

The setup presented here is very simple and will behave like this:

[iSCSI server] -- encrypted data on the server and over the wire -- [iSCSI client]

Note: all these instructions are valid for FreeBSD 7.0 - previous versions are probably missing some parts.

Setting up an iSCSI target

You can skip this section if you already have an iSCSI target (a "target" is where the data is stored,

 i.e. the "server" node of iSCSI).

1. Install the iscsi-target port.

2. Edit /usr/local/etc/iscsi/targets file and add lines similar to the following:

# NAME          DEVICE          START           LENGTH
extent0         /dev/da2        0               5GB
# NAME          ACCESS          STORAGE         NETMASK
target0         rw              extent0         10.0.0.0/24

These lines should be self-explanatory. If you need more help, see targets(5) or NetBSD's iscsi-target HOWTO.

3. Enable iscsi-target in /etc/rc.conf by adding the following line to it:

iscsi_target_enable="YES"

4. Start the server by running /usr/local/etc/rc.d/iscsi_target start. You should see something like the

 following outputted to the console:

Starting iscsi_target.
Reading configuration from `/usr/local/etc/iscsi/targets'
target0:rw:10.0.0.0/24
        extent0:/dev/da2:0:5368709120
DISK: 1 logical unit (10485760 blocks, 512 bytes/block), type iscsi fs
DISK: LUN 0: 5120 MB disk storage for "target0"
TARGET: TargetName is iqn.1994-04.org.netbsd.iscsi-target

Setting up the iSCSI initiator

The "initiator" is the client part in iSCSI, and it connects to the server. The following steps

should be done on the client system.

1. Edit /etc/iscsi.conf and add the following lines:

target0 { # nickname
        targetaddress        = 10.0.0.102
        targetname           = iqn.1994-04.org.netbsd.iscsi-target:target0
}

2. Load the iscsi_initiator kernel module with:

# kldload iscsi_initiator

Also, add the following line to /etc/loader.conf to load the module on boot:

iscsi_initiator_load="YES"

3. Start the iSCSI session by running:

# iscontrol -n target0

Several lines should be output to the console, which should look like the following:

iscontrol[8516]: running
iscontrol[8516]: (pass3:iscsi0:0:0:0):  tagged openings now 0
iscontrol[8516]: cam_open_btl: no passthrough device found at 1:0:1
iscontrol[8516]: cam_open_btl: no passthrough device found at 1:0:2
iscontrol[8516]: cam_open_btl: no passthrough device found at 1:0:3
iscontrol: supervise starting main loop

More importantly, the kernel log (which you can see with tail /var/log/messages) should now contain something similar to this output:

Jan  4 23:17:08 client kernel: da0 at iscsi0 bus 0 target 0 lun 0
Jan  4 23:17:08 client kernel: da0:  Fixed Direct Access SCSI-3 device

This means the device da0 has been created - this is the local representation of the remote iSCSI drive. Technically, da0 is the GEOM device node for a SCSI-like storage device. All further transformations on it are performed as natural parts of the GEOM framework.

3. Set up GEOM_GELI on the new device:

# geli init /dev/da0

The utility will ask you for a passphrase which will be used to encrypt the data. GEOM_ELI (as is the encryption layer known) has many more options, but the defaults are good enough. It will use AES encryption with sane defaults.

4. Load the GEOM_ELI kernel module:

# kldload geom_eli.ko

Also, add the following to /boot/loader.conf to load the module at boot time:

geom_eli_load="YES"

5. Attach the encrypted device:

# geli attach /dev/da0

Lines similar to the following should appear in the kernel log:

Jan  4 23:33:28 client kernel: GEOM_ELI: Device da0.eli created.
Jan  4 23:33:28 client kernel: GEOM_ELI: Encryption: AES-CBC 128
Jan  4 23:33:28 client kernel: GEOM_ELI:     Crypto: software

The device da0.eli has been created - this is the end-point device that can be used by file systems and for other purposes (swap, etc.).

6. Make the file system and mount it!

# newfs -U -L mydata /dev/da0.eli

A successful run of newfs looks something like this:

/dev/da0.eli: 5120.0MB (10485756 sectors) block size 16384, fragment size 2048
        using 28 cylinder groups of 183.77MB, 11761 blks, 23552 inodes.
        with soft updates
super-block backups (for fsck -b #) at:
 160, 376512, 752864, 1129216, 1505568, 1881920, 2258272, 2634624, 3010976, 3387328,
 3763680, 4140032, 4516384, 4892736, 5269088, 5645440, 6021792, 6398144,
 6774496, 7150848, 7527200, 7903552, 8279904, 8656256, 9032608, 9408960, 9785312, 
 10161664

Since we used a volume label for the file system, observe the following message in the kernel log:

Jan  4 23:38:17 client kernel: GEOM_LABEL: Label for provider da0.eli is ufs/mydata.

Now you can mount the file system:

# mount /dev/ufs/mydata /mydata

And that's it!

There are two points that can't be readily automated right now: the iscontrol step which starts the iSCSI initiator, and the geli requiring a password. The former can be approximated by creating a small shell script that does the step and putting it in /usr/local/etc/rc.d but the second cannot be, since the whole point of having an encrypted storage is that it isn't accessible by unwanted people.

The way this setup works is that the unencrypted data is used by the file system (as it should - you wouldn't be able to use it otherwise) via the da0.eli device. This data is encrypted and the encrypted data is written to da0 device. This is the iSCSI client device and the data is tranferred to the server in its encrypted form. The server and the network never see unencrypted data.