Ceph Luminous手动解决pg分布不均衡问题
原文链接:
https://www.jianshu.com/p/afb6277dbfd6
1.设置集群仅支持 Luminous(或者L之后的)客户端
具体命令:
ceph osd set-require-min-compat-client luminous
或
ceph osd set-require-min-compat-client luminous --yes-i-really-mean-it
ceph osd set-require-min-compat-client luminous
或
ceph osd set-require-min-compat-client luminous --yes-i-really-mean-it
[root@controller ~]# ceph features { "mon": [ { "features": "0x3ffddff8ffacffff", "release": "luminous", "num": 1 } ], "osd": [ { "features": "0x3ffddff8ffacffff", "release": "luminous", "num": 14 } ], "client": [ { "features": "0x3ffddff8ffacffff", "release": "luminous", "num": 20 } ], "mgr": [ { "features": "0x3ffddff8ffacffff", "release": "luminous", "num": 1 } ] }
2.获取集群当前的osdmap信息
命令
ceph osd getmap {osdmap_filename}
[root@controller ~]# ceph osd getmap -o osd.map got osdmap epoch 52619
3.获取当前集群数据均衡后的优化信息
命令
osdmaptool {osdmap_filename} --upmap out.txt [--upmap-pool <pool>] [--upmap-max <max-count>] [--upmap-deviation <max-deviation>]
其中
-
upmap-pool :指定需要优化均衡的存储池名
-
upmap-max: 指定一次优化的数据条目,默认100,可根据环境业务情况调整该值,一次调整的条目越多,数据迁移会越多,可能对环境业务造成影响。
-
max-deviation:最大偏差值,默认为0.01(即1%)。如果OSD利用率与平均值之间的差异小于此值,则将被视为完美。
假设只优化2条,即:
[root@controller ~]# osdmaptool osd.map --upmap out.txt --upmap-pool volumes --upmap-max=2 osdmaptool: osdmap file 'osd.map' writing upmap command output to: out.txt checking for upmap cleanups upmap, max-count 2, max deviation 0.01 limiting to pools volumes (2)
[root@controller ~]# cat out.txt ceph osd pg-upmap-items 2.2de 10 5 ceph osd pg-upmap-items 2.2e1 8 5
从上述数据结果中
ceph osd pg-upmap-items 2.2de 10 5
表明pg 12.2de需从osd.10重新映射到osd.5
4.均衡
[root@controller ~]# set 2.2de pg_upmap_items mapping to [10->5] [root@controller ~]# set 2.2d1 pg_upmap_items mapping to [8->5]