ceph存储osd启动异常处理和正常启停操作

机器角色:cloudstack虚拟机的宿主机;ceph存储机器。

事件:ceph存储的物理机器由于内存异常,需要停机更换,仅仅是把该物理机上面的虚拟机迁移走,同时启动了停机维护,然后就直接关机。结果造成重启之后ceph异常

 

原因:由于异常关闭,ceph进程的相关信息没有正常关闭,信息没有同步到文件系统,如pid文件等信息

 

现象并尝试解决:

1)检查osd的整体信息

[root@haha1~]# ceph osd tree

ID WEIGHT    TYPE NAME             UP/DOWN REWEIGHT PRIMARY-AFFINITY

-1 101.91998 root default                                           

-2  25.48000     host haha-50                                  

 1   3.64000         osd.1              up  1.00000          1.00000

 2   3.64000         osd.2              up  1.00000          1.00000

 3   3.64000         osd.3              up  1.00000          1.00000

 4   3.64000         osd.4              up  1.00000          1.00000

 5   3.64000         osd.5              up  1.00000          1.00000

 6   3.64000         osd.6              up  1.00000          1.00000

 0   3.64000         osd.0              up  1.00000          1.00000

-3  25.48000     host XKDHhost1-51                                  

 7   3.64000         osd.7              up  1.00000          1.00000

 9   3.64000         osd.9              up  1.00000          1.00000

10   3.64000         osd.10           down        0          1.00000

11   3.64000         osd.11           down        0          1.00000

12   3.64000         osd.12             up  1.00000          1.00000

13   3.64000         osd.13             up  1.00000          1.00000 

2)osd显示的是down,但是通过

[root@haha1 ~]# /etc/init.d/ceph status osd.11

=== osd.11 ===

osd.11: running {"version":"0.94.2"}

 

3)重启osd.11尝试解决

[root@haha1 ~]# /etc/init.d/ceph restart osd.11

=== osd.11 ===

=== osd.11 ===

Stopping Ceph osd.11 on haha1...kill 7330...kill 7330...done #有kill,可以正常重启

=== osd.11 ===

create-or-move updated item name 'osd.11' weight 3.64 at location {host=XKDHhost1-51,root=default} to crush map

Starting Ceph osd.11 on haha1...

Running as unit run-35058.service.

 

4)osd.10启动异常

[root@haha1 ~]# /etc/init.d/ceph start osd.10

=== osd.10 ===

create-or-move updated item name 'osd.10' weight 3.64 at location {host=haha1,root=default} to crush map

Starting Ceph osd.10 on haha1...

Running as unit run-36525.service.

[root@haha1 ~]# /etc/init.d/ceph status osd.10

=== osd.10 ===

osd.10: not running.

s=a>create-or-move updated item name 'osd.11' weight 3.64 at location {root=default} to crush map

 

Starting Ceph osd.11 on haha1...

Running as unit run-35058.service.

 

 

 

 

 

 

posted @ 2018-01-22 10:05  littlevigra  阅读(989)  评论(4编辑  收藏  举报