nova network-vif-plugged 事件分析1

在创建虚机过程中,nova-compute会调用wait_for_instance_event函数(nova/compute/manage.py)进行network-vif-plugged的事件等待,本文试图介绍这一事件的发生过程,以及超时后的影响。在第一节中主要通过两个简单的实验说明network-vif-plugged事件的作用以及从事件发起到事件处理的流程。

在nova-compute配置文件中有两个与该事件相关的参数- vif_plugging_timeout、vif_plugging_if_fatal,前者是等待事件的最大时间,后者是处理超时异常的方式。若在规定时间内,nova-compute接受到了事件响应,那么虚机可正常创建,那么当超时现象发生时,nova-compute会根据vif_plugging_is_fatal的配置采取两种处理方式。

若超时发生,并且vif_plugging_is_fatal为True,nova首先执行guest.poweroff,停止qemu进程;然后执行cleanup函数,先清除网络资源,使用函数_unplug_vifs,删除plug_vifs函数创建的ovs port,ovs agent检测到了删除端口的事件然后通知neutron-server删除neutron db中的port信息,最后抛VirtualInterfaceCreateException。

如果vif_plugging_is_fatal为False,即便发生eventlet.timeout.Timeout异常,创建过程会继续,这说明network-vif-plugged不会影响虚机的创建,那么该事件的作用是什么?在nova-compute代码中,network-vif-plugged事件并不是通过neutron-client通知的neutron-server,那么neutron-server是如何受到事件请求,并处理事件的呢?

在neutron-server的NeutronDbPluginV2类(neutron/db/db_base_plugin_v2.py)初始化函数中,针对port定义了三类事件及其触发的操作:

event.listen(models_v2.Port,'after_insert', self.nova_notifier.send_port_status)

event.listen(models_v2.Port,'after_update', self.nova_notifier.send_port_status)

event.listen(models_v2.Port.status, 'set', self.nova_notifier.record_port_status_changed)

record_port_status_changed函数用来定义port的_notify_event属性,该属性记录了事件的完成状态。其中的对”set”操作监听,说明在端口的状态被改变时会触发network-vif-plugged事件并调用相应的函数通知nova-compute。那么nova-compute对network-vif-plugged事件的等待是怎么传递到Neutron-server的,我们通过实验就行验证。

实验1 将vif_plugging_is_fatal设置为False,关闭计算节点的neutron-openvswitch-agent服务,启动虚机,虚机可正常启动,但是虚机对应端口状态为Down。

虚拟也拿不到IP地址。

启动ovs-agent后port状态立即变为Active,虚机可以拿到IP。

实验证明,ovs-agent关闭时,无法捕获到nova添加ovs端口的操作,neutron不会更新虚机的端口状态,因此会造成network-vif-plugged事件超时的现象。也就是说network-vif-plugged事件要想触发,必须要由ovs-agent通知neutron-server。

在正常情况下这一流程又是如何发生的呢,我们通过下面的实验进行观察。

实验2 正常情况下,创建虚机时network-vif-plugged的发生和处理。

(1)计算节点的Nova-compute等待执行ovs-vsctl等命令,然后等待network-vif-plugged事件的反馈。

2016-08-16 12:26:41.650 1231 DEBUGoslo_concurrency.processutils [req-8f7c1a9d-7647-4207-824b-ef281a47d14850a809139b434c2287b4f82474242533 630b741632ba47b596e21eaf50f2b568 - - -]Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf ovs-vsctl--timeout=120 -- --if-exists del-port qvo059c0374-b8 -- add-port br-intqvo059c0374-b8 -- set Interface qvo059c0374-b8external-ids:iface-id=059c0374-b801-4ab3-b652-4b33e405eb15external-ids:iface-status=active external-ids:attached-mac=fa:16:3e:3f:68:6eexternal-ids:vm-uuid=bc7d4a87-4df4-413b-9364-39e7780f64a1 execute/usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:344

(2)计算节点的ovs-agent检测到有新端口添加。

12:26:41.776 23624 DEBUGneutron.agent.linux.async_process [-] Output received from [ovsdb-clientmonitor Interface name,ofport,external_ids --format=json]:{"data":[["cf5ad2e5-5adc-463a-845d-dcb156ce1bdc","insert","qvo059c0374-b8",["set",[]],["map",[["attached-mac","fa:16:3e:3f:68:6e"],["iface-id","059c0374-b801-4ab3-b652-4b33e405eb15"],["iface-status","active"],["vm-uuid","bc7d4a87-4df4-413b-9364-39e7780f64a1"]]]]],"headings":["row","action","name","ofport","external_ids"]}_read_stdout/usr/lib/python2.7/site-packages/neutron/agent/linux/async_process.py:236

然后通知Neutron-server。

2016-08-16 12:26:42.161 23624 DEBUGoslo_messaging._drivers.amqpdriver [req-e02bb72d-a5f4-4970-8e3e-961b189a72db -- - - -] CALL msg_id: 4f9c9facb1fa4969889783a986e2c561 exchange 'neutron' topic'q-plugin' _send /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:454

(3)Neutron-server收到信息。

2016-08-16 12:26:42.164 25184 DEBUGoslo_messaging._drivers.amqpdriver [-] received message msg_id:4f9c9facb1fa4969889783a986e2c561 reply toreply_42e9b47bc89d494bac7cd08e1590b22b __call__/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:201

2016-08-16 12:26:42.165 25184 DEBUG neutron.plugins.ml2.rpc[req-e02bb72d-a5f4-4970-8e3e-961b189a72db - - - - -] Device059c0374-b801-4ab3-b652-4b33e405eb15 details requested by agentovs-agent-gansu.saltmaster with host gansu.saltmaster get_device_details/usr/lib/python2.7/site-packages/neutron/plugins/ml2/rpc.py:70

2016-08-16 12:26:42.226 25184 DEBUG neutron.plugins.ml2.db[req-e02bb72d-a5f4-4970-8e3e-961b189a72db - - - - -] For port059c0374-b801-4ab3-b652-4b33e405eb15, host gansu.saltmaster, got binding levels[<neutron.plugins.ml2.models.PortBindingLevel[object at 5ebd1d0]{port_id=u'059c0374-b801-4ab3-b652-4b33e405eb15', host=u'gansu.saltmaster',level=0, driver=u'openvswitch',segment_id=u'7c1db2be-4425-4f92-9913-d0169e4afc7b'}>] get_binding_levels/usr/lib/python2.7/site-packages/neutron/plugins/ml2/db.py:184

2016-08-16 12:26:42.249 25184 DEBUG neutron.notifiers.nova[req-e02bb72d-a5f4-4970-8e3e-961b189a72db - - - - -] Ignoring state changeprevious_port_status: DOWN current_port_status: BUILD port_id059c0374-b801-4ab3-b652-4b33e405eb15 record_port_status_changed/usr/lib/python2.7/site-packages/neutron/notifiers/nova.py:194

(4)Neutron-server将端口的状态由Down转变为Actice并向nova回复事件。

2016-08-16 12:26:44.634 25188 DEBUG neutron.notifiers.nova[-] Sending events: [{'status': 'completed', 'tag':u'059c0374-b801-4ab3-b652-4b33e405eb15', 'name': 'network-vif-plugged','server_uuid': u'bc7d4a87-4df4-413b-9364-39e7780f64a1'}] send_events/usr/lib/python2.7/site-packages/neutron/notifiers/nova.py:209

(5)在nova-api日志中可查看到

2016-08-16 12:26:44.822 31556 DEBUG nova.api.openstack.wsgi[req-efb87df1-0901-435f-8ba4-d7461794ef56 1333f6ff00fa428c984b3ba0a99f08440de50e7e0c7c4f148300936580762205 - - -] Action: 'create', calling method:<bound method ServerExternalEventsController.create of<nova.api.openstack.compute.server_external_events.ServerExternalEventsControllerobject at 0x7afd0d0>>, body: {"events": [{"status":"completed", "tag":"059c0374-b801-4ab3-b652-4b33e405eb15", "name":"network-vif-plugged", "server_uuid":"bc7d4a87-4df4-413b-9364-39e7780f64a1"}]} _process_stack/usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py:696

注:neutron-server在notify的时候调用了nova-client server_external_events的create功能,向nova-api发送了一个create请求。(大致如此,细节忘了J)

2016-08-16 12:26:44.879 31556 INFOnova.api.openstack.compute.server_external_events [req-efb87df1-0901-435f-8ba4-d7461794ef561333f6ff00fa428c984b3ba0a99f0844 0de50e7e0c7c4f148300936580762205 - - -]Creating event network-vif-plugged:059c0374-b801-4ab3-b652-4b33e405eb15 forinstance bc7d4a87-4df4-413b-9364-39e7780f64a1

(6)nova-compute收到事件的回复

2016-08-16 12:26:45.050 1231 DEBUG nova.compute.manager[req-efb87df1-0901-435f-8ba4-d7461794ef56 1333f6ff00fa428c984b3ba0a99f08440de50e7e0c7c4f148300936580762205 - - -] [instance:bc7d4a87-4df4-413b-9364-39e7780f64a1] Received eventnetwork-vif-plugged-059c0374-b801-4ab3-b652-4b33e405eb15external_instance_event/usr/lib/python2.7/site-packages/nova/compute/manager.py:6707

(7)neutron-server得到nova-api的回复

12:26:45.056 25188 INFO neutron.notifiers.nova [-] Nova eventresponse: {u'status': u'completed', u'tag': u'059c0374-b801-4ab3-b652-4b33e405eb15',u'name': u'network-vif-plugged', u'server_uuid':u'bc7d4a87-4df4-413b-9364-39e7780f64a1', u'code': 200} 

 

posted @ 2018-05-30 16:55  salami_china  阅读(874)  评论(0编辑  收藏  举报