ceph多集群对接openstack后端
1、SJ-6-Cloud121 192.168.170.121 操作
创建ceph 磁盘类型
cinder type-create sata02
cinder type-key sata02 set volume_backend_name=CEPH02_SATA
cinder type-create sata03
cinder type-key sata03 set volume_backend_name=CEPH03_SATA
配置cinder文件
mkdir /etc/ceph/ceph02/
mkdir /etc/ceph/ceph03/
scp 192.168.170.36:/etc/ceph/ceph.conf /etc/ceph/ceph02/
scp 192.168.170.36:/etc/ceph/ceph.client.admin.keyring /etc/ceph/ceph02/
scp 192.168.170.41:/etc/ceph/ceph.conf /etc/ceph/ceph03/
scp 192.168.170.41:/etc/ceph/ceph.client.admin.keyring /etc/ceph/ceph03/
enabled_backends=sata02,sata03
[sata02]
volume_driver=cinder.volume.drivers.rbd.RBDDriver
volume_backend_name=CEPH02_SATA
rbd_pool=volumes
rbd_ceph_conf=/etc/ceph/ceph02/ceph.conf
rbd_user=cinder
rbd_secret_uuid=62b1ed02-5de2-4a53-be1e-0477d1b385ae
rbd_max_clone_depth=10
rbd_flatten_volume_from_snapshot = false
rbd_store_chunk_size=4
[sata03]
volume_driver=cinder.volume.drivers.rbd.RBDDriver
volume_backend_name=CEPH03_SATA
rbd_pool=volumes
rbd_ceph_conf=/etc/ceph/ceph03/ceph.conf
rbd_user=cinder
rbd_secret_uuid=62b1ed02-5de2-4a53-be1e-0477d1b385ae
rbd_max_clone_depth=10
rbd_flatten_volume_from_snapshot = false
rbd_store_chunk_size=4
192.168.170.121 获取cinder glance key
获取默认ceph cinder glance 认证key
scp /ect/ceph/ceph.client.cinder.keyring 192.168.170.36:/etc/ceph/
scp /ect/ceph/ceph.client.glance.keyring 192.168.170.36:/etc/ceph/
ssh 192.168.170.36
导入ceph02 auth key
删除
ceph auth import -i ceph.client.cinder.keyring
ceph auth import -i ceph.client.glance.keyring
更新权限ceph
ceph auth caps client.glance mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=volumes, allow rwx pool=images'
192.168.170.121 获取cinder glance key
获取默认ceph cinder glance 认证key
scp /ect/ceph/ceph.client.cinder.keyring 192.168.170.41:/etc/ceph/
scp /ect/ceph/ceph.client.glance.keyring 192.168.170.41:/etc/ceph/
ssh 192.168.170.41
导入ceph03 auth key
删除
ceph auth import -i ceph.client.cinder.keyring
ceph auth import -i ceph.client.glance.keyring
更新权限ceph
ceph auth caps client.glance mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=volumes, allow rwx pool=images'
重启cinder服务服务
openstack-service restart openstack-cinder-api
openstack-service restart openstack-cinder-scheduler
openstack-service restart openstack-cinder-volume
配置完此处之后，glance从镜像启动创建磁盘时，rabbitmq消息，多个后端cinder-volume都有可能接收到；此时应该指定默认的类型来接收；
1 故障 2 3 消息 4 Build of instance 9865bdf5-12e5-4147-b1cf-d0e33a5ae88d aborted: Failure prepping block device. 5 编码 6 500 7 详情 8 File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2235, in _do_build_and_run_instance filter_properties) File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2349, in _build_and_run_instance 'create.error', fault=e) File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__ six.reraise(self.type_, self.value, self.tb) File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2319, in _build_and_run_instance block_device_mapping) as resources: File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ return self.gen.next() File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2458, in _build_resources reason=msg)
cinder配置多ceph后端：并指定默认pool
创建type： cinder type-create XXX
cinder type-list
+-------------------------------------------------------+--------+
| ID | Name |
+-------------------------------------------------------+--------+
| 1908f00c-a865-472c-8f90-c44d29361bd3 | sata03 |
| 3b6e978f-839b-4164-b5e5-8e6e2d63fdde | sata |
| ead23626-0bb9-4515-9906-0b5e6f41320a | sata02 |
+-------------------------------------------------------+--------+
指定extra-specs：cinder type-key XXX set volume_backend_name=CEPH_XXX
cinder extra-specs-list
+------------------------------------------------------------+--------+------------------------------------------------------------+
| ID | Name | extra_specs |
+------------------------------------------------------------+--------+------------------------------------------------------------+
| 1908f00c-a865-472c-8f90-c44d29361bd3 | sata03 | {u'volume_backend_name': u'CEPH03_SATA'} |
| 3b6e978f-839b-4164-b5e5-8e6e2d63fdde | sata | {u'volume_backend_name': u'RBD_SATA'} |
| ead23626-0bb9-4515-9906-0b5e6f41320a | sata02 | {u'volume_backend_name': u'CEPH02_SATA'} |
+------------------------------------------------------------+--------+------------------------------------------------------------+
修改cinder.conf:
在DEFAULT段下指定default_volume_type = XXX
[DEFAULT]
...
default_volume_type=XXX #glance创建的磁盘将位于此处
enabled_backends=sata02, XXX
[XXX]
volume_driver=cinder.volume.drivers.rbd.RBDDriver
volume_backend_name=CEPH_XXX
rbd_pool=volumes
rbd_ceph_conf=/etc/ceph/ceph02/ceph.conf
rbd_user=cinder
rbd_secret_uuid=62b1ed02-5de2-4a53-be1e-0477d1b385ae
rbd_max_clone_depth=10
rbd_flatten_volume_from_snapshot = false
rbd_store_chunk_size=4
