Ceph-RBD

1、创建RBD

创建存储池

 $ ceph osd pool create myrbd 64 64

• myrbd 存储池名称
• 64 是PG数
• 64 是PGP数，pgp是对存在于pg的数据进行组合的存储，一般等于pg数

启用块存储

$ ceph osd pool application enable myrbd rbd

初始化

$  rbd pool init -p myrbd

创建并验证img

rbd存储池不能直接用于块设备，而是需要事先创建映像，并把映像作为块设备使用。

创建

$  rbd create myimg --size 5G --pool myrbd

指定一些特性创建（由于有些内核不支持一些默认特性，所以要根据实际情况选择新特性）

$ rbd create myimg1 --size 5G --pool myrbd --image-format 2 --image-feature layering

查看img

$ rbd ls --pool myrbd
myimg
myimg1

$ rbd --image myimg --pool myrbd info
rbd image 'myimg':
    size 5 GiB in 1280 objects
    order 22 (4 MiB objects)
    snapshot_count: 0
    id: 5e3ac13f7b1c
    block_name_prefix: rbd_data.5e3ac13f7b1c
    format: 2
    features: layering, exclusive-lock, object-map, fast-diff, deep-flatten
    op_features:
    flags:
    create_timestamp: Tue Aug 24 02:27:04 2021
    access_timestamp: Tue Aug 24 02:27:04 2021
    modify_timestamp: Tue Aug 24 02:27:04 2021

 客户端安装ceph-common(我的客户端版本是ubuntu16.04，只能安装N版本ceph-common)

#添加源
$ echo "deb https://mirrors.tuna.tsinghua.edu.cn/ceph/debian-nautilus xenial main" >> /etc/apt/sources.list
#安装ceph-common
$ apt-get install ceph-common

#deploy节点copy  认证文件和配置文件
scp ceph.client.admin.keyring ceph.conf root@172.16.143.175:/etc/ceph

#验证是否正常，客户端执行ceph -s
$ ceph -s
  cluster:
    id:     6e278817-8019-4a06-82b3-b4d24d7dd743
    health: HEALTH_OK

  services:
    mon: 3 daemons, quorum ceph-mon1,ceph-mon2,ceph-mon3 (age 20h)
    mgr: ceph-mgr1(active, since 4d), standbys: ceph-mgr2
    osd: 8 osds: 8 up (since 22h), 8 in (since 22h)

  data:
    pools:   2 pools, 65 pgs
    objects: 7 objects, 405 B
    usage:   49 MiB used, 800 GiB / 800 GiB avail
    pgs:     65 active+clean

升级客户端内核（自行解决）

#客户端映射img
$ rbd map myrbd/myimg1
#查看结果
$ lsblk
NAME                  MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
rbd0                  252:0    0     5G  0 disk
sr0                    11:0    1  1024M  0 rom
rbd1                  252:16   0     5G  0 disk
sda                     8:0    0   200G  0 disk
├─sda2                  8:2    0     1K  0 part
├─sda5                  8:5    0 199.3G  0 part
│ ├─ubuntu--vg-swap_1 253:1    0   980M  0 lvm
│ │ └─cryptswap1      253:2    0 979.5M  0 crypt [SWAP]
│ └─ubuntu--vg-root   253:0    0 198.3G  0 lvm   /
└─sda1                  8:1    0   731M  0 part  /boot

#格式化磁盘
$ mkfs.xfs /dev/rbd0
#挂载目录
$ mount /dev/rbd0 /root/data/
#查看是否挂载
$ df -h
Filesystem                   Size  Used Avail Use% Mounted on
udev                         3.9G     0  3.9G   0% /dev
tmpfs                        798M   19M  779M   3% /run
/dev/mapper/ubuntu--vg-root  196G  7.1G  179G   4% /
tmpfs                        3.9G     0  3.9G   0% /dev/shm
tmpfs                        5.0M     0  5.0M   0% /run/lock
tmpfs                        3.9G     0  3.9G   0% /sys/fs/cgroup
/dev/sda1                    720M  176M  508M  26% /boot
tmpfs                        798M     0  798M   0% /run/user/0
overlay                      196G  7.1G  179G   4% /var/lib/docker/overlay2/437540fd59af26b759cbb0d529c7441988af7b5481de9bc526048c303ba2b42f/merged
overlay                      196G  7.1G  179G   4% /var/lib/docker/overlay2/0a97fd71931b39f5080102748bd8ead60883fe5ca1dc293ed2d16eb6a388e8ed/merged
overlay                      196G  7.1G  179G   4% /var/lib/docker/overlay2/3aac1806df6c62da13c7bb91160d1324b66a4939f69a7e7dc262a3d9928bb788/merged
shm                           64M     0   64M   0% /var/lib/docker/containers/ae6549bfee1770dfb6b86cd0b3a4d2dcea9cd5219868f6f7539dda359974d4f7/mounts/shm
shm                           64M     0   64M   0% /var/lib/docker/containers/4f0ab7d32749845510d351701584b50ce9e340724714c6e5f3564413febc11bb/mounts/shm
shm                           64M   20K   64M   1% /var/lib/docker/containers/ea0deff0be06419cc7ace76f1b810ca18df3ec2ac6d9c19155643e940a31971f/mounts/shm
overlay                      196G  7.1G  179G   4% /var/lib/docker/overlay2/7c40806e67270cf5f395ea30c606e51e846e292ecc0c22a8aa2b2ff215fc1e49/merged
shm                           64M     0   64M   0% /var/lib/docker/containers/889c671b74d8356116f6dbeaf709e06399105f0b6ce41141d4530187b1a275fb/mounts/shm
/dev/rbd0                    5.0G   38M  5.0G   1% /root/data