OpenStack删除计算节点后控制台无法获取虚拟机管理器信息
公司平台一台物理节点出故障手动移除该节点后,点击控制台报错如下截图,实际使用中管理台的计算节点功能均正常。
错误:无法获取虚拟机管理信息。
下线物理机的时候原物理机的主机名为computer1新上线物理机为compute1,通过openstack compute service list命令显示如下:
[root@controller1 images]# openstack compute service list
+----+------------------+-------------+----------+---------+-------+----------------------------+
| ID | Binary | Host | Zone | Status | State | Updated At |
+----+------------------+-------------+----------+---------+-------+----------------------------+
| 2 | nova-cert | controller1 | internal | enabled | up | 2017-08-23T03:47:32.000000 |
| 6 | nova-conductor | controller1 | internal | enabled | up | 2017-08-23T03:47:36.000000 |
| 17 | nova-scheduler | controller1 | internal | enabled | up | 2017-08-23T03:47:34.000000 |
| 25 | nova-consoleauth | controller1 | internal | enabled | up | 2017-08-23T03:47:31.000000 |
| 27 | nova-cert | controller2 | internal | enabled | up | 2017-08-23T03:47:36.000000 |
| 30 | nova-cert | controller3 | internal | enabled | up | 2017-08-23T03:47:33.000000 |
| 32 | nova-conductor | controller3 | internal | enabled | up | 2017-08-23T03:47:35.000000 |
| 36 | nova-scheduler | controller3 | internal | enabled | up | 2017-08-23T03:47:38.000000 |
| 39 | nova-scheduler | controller2 | internal | enabled | up | 2017-08-23T03:47:37.000000 |
| 43 | nova-conductor | controller2 | internal | enabled | up | 2017-08-23T03:47:37.000000 |
| 57 | nova-compute | compute1 | nova | enabled | up | 2017-08-23T03:46:50.000000 |
| 59 | nova-compute | compute1 | nova | enabled | down | 2017-08-23T03:46:50.000000 |
+----+------------------+-------------+----------+---------+-------+----------------------------+
使用命令:
[root@controller1 images]# openstack compute service -h
Command "compute" matches:
compute agent create
compute agent delete
compute agent list
compute agent set
compute service delete
compute service list
compute service set
[root@controller1 images]# openstack compute service delete -h
usage: openstack compute service delete [-h] <service> [<service> ...]
Delete compute service(s)
positional arguments:
<service> Compute service(s) to delete (ID only)
optional arguments:
-h, --help show this help message and exit
[root@controller1 images]# openstack compute service delete 59
[root@controller1 images]# openstack compute service list
+----+------------------+-------------+----------+---------+-------+----------------------------+
| ID | Binary | Host | Zone | Status | State | Updated At |
+----+------------------+-------------+----------+---------+-------+----------------------------+
| 2 | nova-cert | controller1 | internal | enabled | up | 2017-08-23T04:04:22.000000 |
| 6 | nova-conductor | controller1 | internal | enabled | up | 2017-08-23T04:04:30.000000 |
| 17 | nova-scheduler | controller1 | internal | enabled | up | 2017-08-23T04:04:24.000000 |
| 25 | nova-consoleauth | controller1 | internal | enabled | up | 2017-08-23T04:04:31.000000 |
| 27 | nova-cert | controller2 | internal | enabled | up | 2017-08-23T04:04:26.000000 |
| 30 | nova-cert | controller3 | internal | enabled | up | 2017-08-23T04:04:24.000000 |
| 32 | nova-conductor | controller3 | internal | enabled | up | 2017-08-23T04:04:32.000000 |
| 36 | nova-scheduler | controller3 | internal | enabled | up | 2017-08-23T04:04:28.000000 |
| 39 | nova-scheduler | controller2 | internal | enabled | up | 2017-08-23T04:04:27.000000 |
| 43 | nova-conductor | controller2 | internal | enabled | up | 2017-08-23T04:04:29.000000 |
| 57 | nova-compute | compute1 | nova | enabled | up | 2017-08-23T04:03:50.000000 |
+----+------------------+-------------+----------+---------+-------+----------------------------+
PS:查询资料建议先使用 nova service-disable computer1 nova-compute禁用掉computer1的nova服务器再进行删除操作。
删除后使用dashboard过程中点击--虚拟机管理器报错:错误:无法获取虚拟机管理信息。
初步怀疑是删除物理节点导致问题,查询/var/log/nova/nova-api.log后发现不能发现主机computer1,初步推断为数据库中有未移除内容导致。
查询数据库中信息:
MariaDB [nova]> select * from nova.services;
+---------------------+---------------------+---------------------+----+-------------+--------------------+-------------+--------------+----------+---------+-------------------------------------+---------------------+-------------+---------+
| created_at | updated_at | deleted_at | id | host | binary | topic | report_count | disabled | deleted | disabled_reason | last_seen_up | forced_down | version |
+---------------------+---------------------+---------------------+----+-------------+--------------------+-------------+--------------+----------+---------+-------------------------------------+---------------------+-------------+---------+
| 2017-08-21 15:58:33 | 2017-08-23 04:23:03 | NULL | 2 | controller1 | nova-cert | cert | 13093 | 0 | 0 | NULL | 2017-08-23 04:23:03 | 0 | 15 |
| 2017-08-21 15:58:33 | 2017-08-23 04:23:10 | NULL | 6 | controller1 | nova-conductor | conductor | 13099 | 0 | 0 | NULL | 2017-08-23 04:23:10 | 0 | 15 |
| 2017-08-21 15:58:34 | NULL | NULL | 15 | 0.0.0.0 | nova-osapi_compute | NULL | 0 | 0 | 0 | NULL | NULL | 0 | 15 |
| 2017-08-21 15:58:35 | 2017-08-23 04:23:04 | NULL | 17 | controller1 | nova-scheduler | scheduler | 13096 | 0 | 0 | NULL | 2017-08-23 04:23:04 | 0 | 15 |
| 2017-08-21 15:58:35 | NULL | NULL | 20 | 0.0.0.0 | nova-metadata | NULL | 0 | 0 | 0 | NULL | NULL | 0 | 15 |
| 2017-08-21 15:58:35 | 2017-08-23 04:23:11 | NULL | 25 | controller1 | nova-consoleauth | consoleauth | 13097 | 0 | 0 | NULL | 2017-08-23 04:23:11 | 0 | 15 |
| 2017-08-21 15:59:00 | 2017-08-23 04:23:07 | NULL | 27 | controller2 | nova-cert | cert | 13088 | 0 | 0 | NULL | 2017-08-23 04:23:07 | 0 | 15 |
| 2017-08-21 15:59:03 | 2017-08-23 04:23:04 | NULL | 30 | controller3 | nova-cert | cert | 13088 | 0 | 0 | NULL | 2017-08-23 04:23:04 | 0 | 15 |
| 2017-08-21 15:59:03 | 2017-08-23 04:23:10 | NULL | 32 | controller3 | nova-conductor | conductor | 13082 | 0 | 0 | NULL | 2017-08-23 04:23:09 | 0 | 15 |
| 2017-08-21 15:59:03 | 2017-08-23 04:23:08 | NULL | 36 | controller3 | nova-scheduler | scheduler | 13094 | 0 | 0 | NULL | 2017-08-23 04:23:08 | 0 | 15 |
| 2017-08-21 15:59:04 | 2017-08-23 04:23:07 | NULL | 39 | controller2 | nova-scheduler | scheduler | 13094 | 0 | 0 | NULL | 2017-08-23 04:23:07 | 0 | 15 |
| 2017-08-21 15:59:05 | 2017-08-23 04:23:10 | NULL | 43 | controller2 | nova-conductor | conductor | 13092 | 0 | 0 | NULL | 2017-08-23 04:23:10 | 0 | 15 |
| 2017-08-21 16:03:34 | 2017-08-22 04:40:22 | 2017-08-22 10:24:43 | 53 | compute1 | nova-compute | compute | 749 | 1 | 53 | AUTO: Connection to libvirt lost: 0 | 2017-08-22 04:39:52 | 0 | 15 |
| 2017-08-22 13:41:19 | 2017-08-23 04:22:50 | NULL | 57 | compute1 | nova-compute | compute | 883 | 0 | 0 | NULL | 2017-08-23 04:22:50 | 0 | 15 |
+---------------------+---------------------+---------------------+----+-------------+--------------------+-------------+--------------+----------+---------+-------------------------------------+--------------
MariaDB [nova]> delete from compute_nodes where hypervisor_hostname="computer1";
Query OK, 2 rows affected (0.56sec)
MariaDB [nova]> delete from nova.services where host="computer1"; //该命令推测与openstack compute service delete XXXXXX功能相同
Query OK, 0 rows affected (0.61 sec)
重新登录dashboard后正常显示节点信息。