openstack havana块存储Cinder磁盘加密方法研究
http://blog.csdn.net/cloudresearch/article/details/19092219
在openstack havana的release note中有如下介绍“Attached Cinder volumes can now be encrypted. Data is decrypted as needed at read and write time while presenting instances with a normal block storage device”。
众所周知，在以前版本的openstack中，块存储创建出volume后，将其挂载给虚拟机，就可以启动虚拟机。但这种实现方式也存在一些安全隐患：若存在不怀好意的系统管理员或者黑客在攻陷nova节点时，就可以从宿主系统中，查看guest os中用户存放的数据。如果用户在其磁盘中存在敏感私人数据时，则存在数据泄密的风险。
关于磁盘加密的BluePrint链接：https://wiki.openstack.org/wiki/VolumeEncryption ，通过分析其实现方式可以发现，cinder中的卷在挂载到HyperVisor主机上时，对其进行加密，再讲加密后的块设备提供给虚拟机使用，最终虚拟机使用的加密后的磁盘。如果此时黑客攻陷主机的话，他看到的将是加密后的volume，如果不知道磁盘的加密密码，他看到将是没有任何意义的一堆数据。
通过分析nova中代码实现，可以发现磁盘加密的实现细节，openstack中的磁盘加密使用了Linux中的一个加密组件：cryptsetup！
- def _open_volume(self, passphrase, **kwargs):
- """Opens the LUKS partition on the volume using the specified
- passphrase.
- :param passphrase: the passphrase used to access the volume
- """
- LOG.debug(_("opening encrypted volume %s"), self.dev_path)
- # NOTE(joel-coffman): cryptsetup will strip trailing newlines from
- # input specified on stdin unless --key-file=- is specified.
- cmd = ["cryptsetup", "create", "--key-file=-"]
- cipher = kwargs.get("cipher", None)
- if cipher is not None:
- cmd.extend(["--cipher", cipher])
- key_size = kwargs.get("key_size", None)
- if key_size is not None:
- cmd.extend(["--key-size", key_size])
- cmd.extend([self.dev_name, self.dev_path])
- utils.execute(*cmd, process_input=passphrase,
- check_exit_code=True, run_as_root=True)
- def attach_volume(self, context, **kwargs):
- """Shadows the device and passes an unencrypted version to the
- instance.
- Transparent disk encryption is achieved by mounting the volume via
- dm-crypt and passing the resulting device to the instance. The
- instance is unaware of the underlying encryption due to modifying the
- original symbolic link to refer to the device mounted by dm-crypt.
- """
- key = self._get_key(context).get_encoded()
- passphrase = self._get_passphrase(key)
- self._open_volume(passphrase, **kwargs)
- # modify the original symbolic link to refer to the decrypted device
- utils.execute('ln', '--symbolic', '--force',
- '/dev/mapper/%s' % self.dev_name, self.symlink_path,
- run_as_root=True, check_exit_code=True)
可见，在attach_volume方法的_open_volume中，会调用cryptsetup系统命令对输入的待挂载设备进行加密，再将加密的设备生成挂载给虚拟机的软链接，这样虚拟机在启动时感知不到加密方法的存在，即磁盘加密对虚拟机是透明的。cryptsetup
由于手头没有havana的环境，下面打算根据cinder的实现原理对磁盘加密进行模拟，同时研究其实现：
1、首先使用truncate命令生成一个空洞文件
- [root@armstrong tmp]# truncate --size=10G 10G
- [root@armstrong tmp]# ll /tmp/10G
- -rw-r--r--. 1 root root 10737418240 12月 3 23:37 /tmp/10G
- [root@armstrong tmp]#
2、将上述空洞文件转化为loop设备
- [root@armstrong tmp]#
- [root@armstrong tmp]# losetup -f /tmp/10G
- [root@armstrong tmp]#
- [root@armstrong tmp]# losetup -a
- /dev/loop0: [0030]:33006 (/tmp/10G)
- [root@armstrong tmp]#
3、在上述loop设备上创建pv、vg
- [root@armstrong tmp]# pvcreate qixiaozhen /dev/loop0
- Device qixiaozhen not found (or ignored by filtering).
- Physical volume "/dev/loop0" successfully created
- [root@armstrong tmp]#
- [root@armstrong tmp]# vgcreate qixiaozhen /dev/loop0
- Volume group "qixiaozhen" successfully created
- [root@armstrong tmp]#
4、创建用以测试用的lv
- [root@armstrong tmp]# lvcreate -n test001 -L 100M qixiaozhen
- Logical volume "test001" created
- [root@armstrong tmp]#
5、使用cryptsetup对上述lv进行加密（注意此处需要用户输入密码）
- [root@armstrong tmp]#
- [root@armstrong tmp]# cryptsetup luksFormat /dev/qixiaozhen/test001
- WARNING!
- ========
- This will overwrite data on /dev/qixiaozhen/test001 irrevocably.
- Are you sure? (Type uppercase yes): YES
- Enter LUKS passphrase:
- Verify passphrase:
- [root@armstrong tmp]#
6、打开加密后的磁盘，生成供虚拟机使用磁盘（需要输入密码）
- [root@armstrong tmp]#
- [root@armstrong tmp]# cryptsetup luksOpen /dev/qixiaozhen/test001 qixiaozhen_device
- Enter passphrase for /dev/qixiaozhen/test001:
- [root@armstrong tmp]#
7、在/dev/mapper路径下可以找到上述生成的qixiaozhen_device设备
- [root@armstrong tmp]#
- [root@armstrong tmp]# ll /dev/mapper/qixiaozhen_device
- lrwxrwxrwx. 1 root root 7 12月 3 23:48 /dev/mapper/qixiaozhen_device -> ../dm-4
- [root@armstrong tmp]#
- [root@armstrong tmp]#
8、分析逻辑卷test001与qixiaozhen_device设备间的关系
- [root@armstrong tmp]#
- [root@armstrong tmp]# dmsetup table
- fedora_armstrong-swap: 0 12255232 linear 8:9 2048
- fedora_armstrong-root: 0 104857600 linear 8:9 995198976
- qixiaozhen-test001: 0 204800 linear 7:0 2048
- qixiaozhen_device: 0 200704 crypt aes-cbc-essiv:sha256 0000000000000000000000000000000000000000000000000000000000000000 0 253:3 4096
- fedora_armstrong-home: 0 982941696 linear 8:9 12257280
- [root@armstrong tmp]#
可见qixiaozhen_device设备是由test001设备偏移4096个扇区（2MB）而生成。
9、往qixiaozhen_device中写入部分数据，本文中通过格式化成ext4文件系统进行测试
- [root@armstrong tmp]# mkfs.ext4 /dev/mapper/qixiaozhen_device
- mke2fs 1.42.5 (29-Jul-2012)
- 文件系统标签=
- OS type: Linux
- 块大小=1024 (log=0)
- 分块大小=1024 (log=0)
- Stride=0 blocks, Stripe width=0 blocks
- 25168 inodes, 100352 blocks
- 5017 blocks (5.00%) reserved for the super user
- 第一个数据块=1
- Maximum filesystem blocks=67371008
- 13 block groups
- 8192 blocks per group, 8192 fragments per group
- 1936 inodes per group
- Superblock backups stored on blocks:
- 8193, 24577, 40961, 57345, 73729
- Allocating group tables: 完成
- 正在写入inode表: 完成
- Creating journal (4096 blocks): 完成
- Writing superblocks and filesystem accounting information: 完成
- [root@armstrong tmp]#
10、下面分析使用dd命令读取test001偏移2MB和qixiaozhen_device进行比较。
- [root@armstrong tmp]# dd if=/dev/qixiaozhen/test001 bs=1M count=1 iflag=direct skip=2 | hexdump -C | less
- 00000000 4c a7 db 72 01 e4 18 1e bc 1e b9 bf fa 3b 25 01 |L..r.........;%.|
- 00000010 cc 22 46 96 aa b9 e8 2c 53 60 7c b3 61 4a 02 33 |."F....,S`|.aJ.3|
- 00000020 5d 39 4b 4f 6d b9 5a 57 16 fb a5 f9 c8 7f 9d 65 |]9KOm.ZW.......e|
- 00000030 3d 25 38 f5 4b e2 b0 6e ef 92 24 bd fc cf 56 07 |=%8.K..n..$...V.|
- 00000040 4b 5f d9 85 ad f4 01 5d dd a3 94 db 8b 58 0f a0 |K_.....].....X..|
- 00000050 03 0f 08 ad 7a 66 a8 3a 72 16 47 58 0c ba f9 d0 |....zf.:r.GX....|
- 00000060 60 34 b4 ba 69 32 27 8c f8 97 cd 58 86 3e ce 34 |`4..i2'....X.>.4|
- 00000070 ae 28 4f e1 c8 a9 90 5c 08 f9 30 9f a7 4b 6b 21 |.(O....\..0..Kk!|
- 00000080 ba d0 39 45 7a 0d a4 c5 a6 c9 a8 40 a2 56 c9 27 |..9Ez......@.V.'|
- 00000090 9d d4 77 05 77 ae b0 2a eb fa 3a 1d 1b d0 19 9d |..w.w..*..:.....|
- 000000a0 fe c3 d9 1c ca 3c 51 fa 84 d7 d8 a8 8d 42 1f e0 |.....<Q......B..|
- 000000b0 bd b0 09 84 69 25 6d 70 9f b6 78 25 87 52 07 70 |....i%mp..x%.R.p|
- 000000c0 e1 9c d9 9c 42 2d 1d fb 92 fd 9d 62 d3 58 27 79 |....B-.....b.X'y|
- 000000d0 2c b3 a5 a8 67 7d 3e 5d 01 15 3d ac 7e 44 77 67 |,...g}>]..=.~Dwg|
- 000000e0 8f 4f 71 e9 4e 63 06 4d 9a 78 7c 0f 60 4a da 3b |.Oq.Nc.M.x|.`J.;|
- 000000f0 0d f2 46 ac c0 c7 44 0c 61 dc 6c ef 00 c0 7b 9f |..F...D.a.l...{.|
- 00000100 60 55 94 7f 1f c2 87 1b 10 93 c0 78 92 6a 8f bf |`U.........x.j..|
- 00000110 6d 08 b1 b7 51 86 6e 83 46 26 cb fe 0d 0f c4 72 |m...Q.n.F&.....r|
- 00000120 b1 48 88 c5 05 be a9 f1 8a ac a2 ed f6 73 40 f5 |.H...........s@.|
- 00000130 74 51 76 ba f4 db 8c be f8 9d 73 14 e8 7b 8c 0f |tQv.......s..{..|
- 00000140 06 bb 89 f9 e4 2e 81 8e 5f c8 f4 a2 70 27 03 a5 |........_...p'..|
- 00000150 3e 93 80 b1 38 4d ad 12 9f ca 4c e8 01 5b 26 41 |>...8M....L..[&A|
- 00000160 73 f9 3f db 75 c5 d5 d9 c7 25 b9 6d 03 4d 39 98 |s.?.u....%.m.M9.|
- 00000170 72 7e 5f 10 31 be e9 92 fa 5c d5 54 9f 61 65 00 |r~_.1....\.T.ae.|
- 00000180 bd e0 4a 8f 8a 84 b4 41 5d b5 6c 42 f5 d5 ff 7a |..J....A].lB...z|
- 00000190 7f 97 c5 dc 73 74 c8 94 f7 03 f0 bd 8d 1d 69 9e |....st........i.|
- 000001a0 93 c4 52 52 d8 9e 21 6a 74 2d 1e d4 29 1e 1e 92 |..RR..!jt-..)...|
- 000001b0 8a 61 03 9d ed 0c ca c0 e5 33 38 49 d3 7e 5f 94 |.a.......38I.~_.|
- 000001c0 44 f9 ae 5d 5f 9c 5a 51 02 20 77 46 9c 46 23 6b |D..]_.ZQ. wF.F#k|
- 000001d0 8b 5e 15 28 4f 09 ce 3e 36 e8 62 21 e4 08 43 e9 |.^.(O..>6.b!..C.|
- 000001e0 b3 53 90 e2 68 fb 28 4d fa 24 e8 7a 3a ce b1 0a |.S..h.(M.$.z:...|
- 000001f0 f8 f1 ae a0 c9 57 82 9d 0e 56 6e f8 d9 d5 81 f8 |.....W...Vn.....|
- 00000200 9d f6 fa 8b d2 f2 bd cf d3 f0 ff e8 52 1f d1 f3 |............R...|
- 00000210 c1 aa ed db 78 55 f8 e3 7c 85 bf ec d2 a3 15 ec |....xU..|.......|
- 00000220 8b 6d 04 cf 2d 14 37 1b f4 22 06 5f c1 8d eb df |.m..-.7.."._....|
- 00000230 77 7b b9 f2 36 f7 b2 63 ad 23 06 ff 23 a1 b5 dc |w{..6..c.#..#...|
- 00000240 56 67 85 1f a2 f5 59 4c cc a0 c0 97 c1 b1 7e b2 |Vg....YL......~.|
- 00000250 e5 7f 6e 3c d2 ee 65 fa 52 10 ea b8 fb e0 38 e8 |..n<..e.R.....8.|
- 00000260 cf 95 9e 80 72 e4 5b 22 2b 9f 72 f6 22 2f e1 f7 |....r.["+.r."/..|
- 00000270 ec f4 49 cf c3 d2 f4 34 de 00 55 17 e9 1c cf 5a |..I....4..U....Z|
- 00000280 19 7f a0 f4 48 14 f5 ee 2f 75 76 ca 0e 94 9f 16 |....H.../uv.....|
- 00000290 51 aa 80 03 a9 e6 2c 69 31 d4 35 d5 d5 b7 e9 58 |Q.....,i1.5....X|
- 000002a0 bc a1 f5 8f 56 bc 12 94 9e d7 d8 cc 7d f7 61 bd |....V.......}.a.|
- [root@armstrong tmp]# dd if=/dev/mapper/qixiaozhen_device bs=1M count=1 iflag=direct | hexdump -C | less
- 00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
- *
- 00000400 50 62 00 00 00 88 01 00 99 13 00 00 81 65 01 00 |Pb...........e..|
- 00000410 45 62 00 00 01 00 00 00 00 00 00 00 00 00 00 00 |Eb..............|
- 00000420 00 20 00 00 00 20 00 00 90 07 00 00 00 00 00 00 |. ... ..........|
- 00000430 8c ff 9d 52 00 00 ff ff 53 ef 01 00 01 00 00 00 |...R....S.......|
- 00000440 8c ff 9d 52 00 00 00 00 00 00 00 00 01 00 00 00 |...R............|
- 00000450 00 00 00 00 0b 00 00 00 80 00 00 00 3c 00 00 00 |............<...|
- 00000460 42 02 00 00 79 00 00 00 0e 6c 42 94 c7 e6 4b 18 |B...y....lB...K.|
- 00000470 99 55 8d 81 b6 2c 5a 9a 00 00 00 00 00 00 00 00 |.U...,Z.........|
- 00000480 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
- *
- 000004c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................|
- 000004d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
- 000004e0 08 00 00 00 00 00 00 00 00 00 00 00 25 2a 68 6f |............%*ho|
- 000004f0 14 38 4a d7 b2 35 fb 62 20 5c ec 36 01 01 00 00 |.8J..5.b \.6....|
- 00000500 0c 00 00 00 00 00 00 00 8c ff 9d 52 0a f3 01 00 |...........R....|
- 00000510 04 00 00 00 00 00 00 00 00 00 00 00 00 10 00 00 |................|
- 00000520 01 c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
- 00000530 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
- 00000540 00 00 00 00 00 00 00 00 00 00 00 00 00 00 40 00 |..............@.|
- 00000550 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
- 00000560 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
- 00000570 00 00 00 00 04 00 00 00 5f 11 00 00 00 00 00 00 |........_.......|
- 00000580 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
- *
- 00000800 03 01 00 00 13 01 00 00 23 01 00 00 8c 12 85 07 |........#.......|
- 00000810 02 00 00 00 00 00 00 00 00 00 00 00 85 07 b9 cc |................|
- 00000820 04 01 00 00 14 01 00 00 15 02 00 00 fe 1e 90 07 |................|
- 00000830 00 00 01 00 00 00 00 00 00 00 00 00 90 07 f4 ce |................|
- 00000840 05 01 00 00 15 01 00 00 07 03 00 00 00 20 90 07 |............. ..|
- 00000850 00 00 03 00 00 00 00 00 00 00 00 00 90 07 77 60 |..............w`|
- 00000860 06 01 00 00 16 01 00 00 f9 03 00 00 fe 1e 90 07 |................|
- 00000870 00 00 01 00 00 00 00 00 00 00 00 00 90 07 36 89 |..............6.|
- 00000880 07 01 00 00 17 01 00 00 eb 04 00 00 00 20 90 07 |............. ..|
- 00000890 00 00 03 00 00 00 00 00 00 00 00 00 90 07 d2 35 |...............5|
- 000008a0 08 01 00 00 18 01 00 00 dd 05 00 00 fe 1e 90 07 |................|
- 000008b0 00 00 01 00 00 00 00 00 00 00 00 00 90 07 e0 b7 |................|
- 000008c0 09 01 00 00 19 01 00 00 cf 06 00 00 00 10 90 07 |................|
- 000008d0 00 00 01 00 00 00 00 00 00 00 00 00 90 07 ad 33 |...............3|
- 000008e0 0a 01 00 00 1a 01 00 00 c1 07 00 00 fe 1e 90 07 |................|
- 000008f0 00 00 01 00 00 00 00 00 00 00 00 00 90 07 ee 88 |................|
- 00000900 0b 01 00 00 1b 01 00 00 b3 08 00 00 00 20 90 07 |............. ..|
结论：可见，直接在主机上读取裸设备上的裸数据时，从加密前设备和加密后设备上的是完全不同的。使用磁盘加密的方法，在删除磁盘时，不需要对磁盘进行格式化，直接把加密用的密码删除即可，再也不用往裸设备中填0实现格式化。
