Cinder（二）：Cinder组件详解：cinder-api、cinder-scheduler、cinder-volume + Volume Provider；Driver 架构、cinder-volume功能
cinder-api： cinder-api 是整个 Cinder 组件的门户，所有 cinder 的请求都首先由 cinder-api 处理。cinder-api 向外界暴露若干 HTTP REST API 接口。 openstack endpoint show cinder #在 keystone 中我们可以查询 cinder-api 的 endponits，该命令会报错 cinder-api 对接收到的 HTTP API 请求会做如下处理： 1.检查客户端传人的参数是否合法有效 2.调用 cinder 其他子服务的处理客户端请求 3.将 cinder 其他子服务返回的结果序列号并返回给客户端
cinder-scheduler
cinder-scheduler；cinder-scheduler配置；scheduler调度过程 ============================================================================ cinder-scheduler： 创建 Volume 时，cinder-scheduler 会基于容量、Volume Type 等条件选择出最合适的存储节点，然后让其创建 Volume。 ----------------------------------------------------------------------------- cinder-scheduler配置 [root@controller cinder]# cat /etc/cinder/cinder.conf |grep -P "scheduler_driver|scheduler_default_filters|scheduler_default_weighers" #scheduler_default_filters = AvailabilityZoneFilter,CapacityFilter,CapabilitiesFilter #scheduler_default_weighers = CapacityWeigher #scheduler_driver = cinder.scheduler.filter_scheduler.FilterScheduler #Filter scheduler 是 cinder-scheduler 默认的调度器 ---------------------------------------------------------------------------- scheduler 调度过程如下： 1.通过过滤器（filter）选择满足条件的存储节点（运行 cinder-volume） 2.通过权重计算（weighting）选择最优（权重值最大）的存储节点。 ---------------------------------------------------------------------------- scheduler 调度过程如下： 1.通过过滤器（filter）选择满足条件的存储节点（运行 cinder-volume） 当 Filter scheduler 需要执行调度操作时，会让 filter 对存储节点进行判断，filter 返回 True 或者 False。 AvailabilityZoneFilter：为提高容灾性和提供隔离服务，可以将存储节点和计算节点划分到不同的 Availability Zone 中。 #OpenStack 默认有一个命名为“Nova”的 Availability Zone，所有的节点初始都是放在“Nova”中。用户可以根据需要创建自己的 Availability Zone。 CapacityFilter：创建 Volume 时，用户会指定 Volume 的大小。CapacityFilter 的作用是将存储空间不能满足 Volume 创建需求的存储节点过滤掉。 CapabilitiesFilter：不同的 Volume Provider 有自己的特性（Capabilities），比如是否支持 thin provision 等。Cinder 允许用户创建 Volume 时通过 Volume Type 指定需要的 Capabilities。 #admin---Volume---Volume Type可以自定义Volume Type，类似 Nova 的 flavor 2.通过权重计算（weighting）选择最优（权重值最大）的存储节点。 默认为 CapacityWeigher CapacityWeigher 基于存储节点的空闲容量计算权重值，空闲最多的胜出。
cinder-volume + Volume Provider；Driver 架构；cinder-volume功能 ------------------------------------------------------------------------------ cinder-volume + Volume Provider；Driver 架构 cinder-volume 与 volume provider 一起实现 volume 生命周期的管理： cinder-volume 在存储节点上运行，OpenStack 对 Volume 的操作，最后都是交给 cinder-volume 来完成的。 volume provider 直接管理存储设备；cinder-volume 自身并不管理真正的存储设备。 ------------------------------------------------------------------------------ cinder-volume功能： 1.cinder-volume通过 Driver 架构支持多种 Volume Provider 存储节点在配置文件 /etc/cinder/cinder.conf 中用 volume_driver 选项配置使用的driver： [root@controller ~]# cat /etc/cinder/cinder.conf |grep volume_driver volume_driver=cinder.volume.drivers.lvm.LVMVolumeDriver 2.cinder-volume 会定期向 OpenStack 报告计算节点的状态 cat /var/log/cinder/volume.log |grep -Pi "updating" -C5 #这个目录居然在控制节点？ 2021-11-24 22:33:56.061 12515 DEBUG oslo_service.periodic_task [req-0f22e5b3-52d3-4c6c-ba1c-3a3525382668 - - - - -] Running periodic task VolumeManager.publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:217 2021-11-24 22:33:56.062 12515 DEBUG cinder.volume.drivers.lvm [req-0f22e5b3-52d3-4c6c-ba1c-3a3525382668 - - - - -] Updating volume stats _update_volume_stats /usr/lib/python2.7/site-packages/cinder/volume/drivers/lvm.py:198 2021-11-24 22:33:56.063 12515 DEBUG oslo_concurrency.processutils [req-0f22e5b3-52d3-4c6c-ba1c-3a3525382668 - - - - -] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C vgs --noheadings --unit=g -o name,size,free,lv_count,uuid --separator : --nosuffix cinder-volumes execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:379 2021-11-24 22:33:56.324 12515 DEBUG oslo_concurrency.processutils [req-0f22e5b3-52d3-4c6c-ba1c-3a3525382668 - - - - -] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C vgs --noheadings --unit=g -o name,size,free,lv_count,uuid --separator : --nosuffix cinder-volumes" returned: 0 in 0.261s execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:416 2021-11-24 22:33:56.325 12515 DEBUG oslo_concurrency.processutils [req-0f22e5b3-52d3-4c6c-ba1c-3a3525382668 - - - - -] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C lvs --noheadings --unit=g -o vg_name,name,size --nosuffix cinder-volumes execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:379 2021-11-24 22:33:56.562 12515 DEBUG oslo_concurrency.processutils [req-0f22e5b3-52d3-4c6c-ba1c-3a3525382668 - - - - -] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C lvs --noheadings --unit=g -o vg_name,name,size --nosuffix cinder-volumes" returned: 0 in 0.237s execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:416 2021-11-24 22:33:56.563 12515 DEBUG oslo_concurrency.processutils [req-0f22e5b3-52d3-4c6c-ba1c-3a3525382668 - - - - -] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C lvs --noheadings --unit=g -o size,data_percent --separator : --nosuffix /dev/cinder-volumes/cinder-volumes-pool execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:379 2021-11-24 22:33:56.812 12515 DEBUG oslo_concurrency.processutils [req-0f22e5b3-52d3-4c6c-ba1c-3a3525382668 - - - - -] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C lvs --noheadings --unit=g -o size,data_percent --separator : --nosuffix /dev/cinder-volumes/cinder-volumes-pool" returned: 0 in 0.249s execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:416 2021-11-24 22:33:56.813 12515 DEBUG oslo_concurrency.processutils [req-0f22e5b3-52d3-4c6c-ba1c-3a3525382668 - - - - -] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C lvs --noheadings --unit=g -o vg_name,name,size --nosuffix cinder-volumes execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:379 2021-11-24 22:33:57.065 12515 DEBUG oslo_concurrency.processutils [req-0f22e5b3-52d3-4c6c-ba1c-3a3525382668 - - - - -] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C lvs --noheadings --unit=g -o vg_name,name,size --nosuffix cinder-volumes" returned: 0 in 0.252s execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:416 2021-11-24 22:33:57.066 12515 DEBUG cinder.manager [req-0f22e5b3-52d3-4c6c-ba1c-3a3525382668 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:194 #因为在我们的实验环境中存储节点使用的是 LVM，所以在上面的日志看到存储节点通过“vgs”和”lvs”这两个命令获取 LVM 的容量使用信息。 3.实现 volume 生命周期管理 Cinder 对 volume 的生命周期的管理最终都是通过 cinder-volume 完成的，包括 volume 的 create、extend、attach、snapshot、delete 等
Create Volume操作流程 ===================================================================================================================================== Create Volume操作流程: 1.客户（可以是 OpenStack 最终用户，也可以是其他程序）向 API（cinder-api）发送请求：“帮我创建一个 volume”。 2.API 对请求做一些必要处理后，向 Messaging（RabbitMQ）发送了一条消息：“让 Scheduler 创建一个 volume”。 3.Scheduler（cinder-scheduler）从 Messaging 获取到 API 发给它的消息，然后执行调度算法，从若干计存储点中选出节点 A。 4.Scheduler 向 Messaging 发送了一条消息：“让存储节点 A 创建这个 volume”。 5.存储节点 A 的 Volume（cinder-volume）从 Messaging 中获取到 Scheduler 发给它的消息，然后通过 driver 在 volume provider 上创建 volume。 Create Volume 操作相对复杂，可以分为3个部分： 1.cinder-api 的处理过程； 2.cinder-scheduler； 3.cinder-volume 的操作。
1.cinder-api接收创建Volume请求，进行一系列处理，并向rabbitMQ发送请求； ------------------------------------------------------------------------------------------------------------------------------------ 1.客户（可以是 OpenStack 最终用户，也可以是其他程序）向 API（cinder-api）发送请求：“帮我创建一个 volume”。 [root@controller cinder]# cat api.log |grep "Action: 'create'" #快速定位requestID [root@controller ~]# cat /var/log/cinder/api.log |grep "req-e419e8b4-b40f-45af-8777-2330e8600512" 2021-11-28 19:43:38.210 12465 INFO cinder.api.openstack.wsgi [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] POST http://192.168.1.21:8776/v3/f911df6b22d54d34b716efd7a97edf92/volumes #白屏create volume时，cinder.api.openstack.wsgi接收到创建的POST请求 2021-11-28 19:43:38.210 12465 DEBUG cinder.api.openstack.wsgi [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Action: 'create', calling method: create, body: {"volume": {"backup_id": null, "description": "volume1111", "availability_zone": "nova", "source_volid": null, "consistencygroup_id": null, "snapshot_id": null, "size": 1, "name": "volume1111", "imageRef": null, "volume_type": "iscsi", "metadata": {}}} _process_stack /usr/lib/python2.7/site-packages/cinder/api/openstack/wsgi.py:876 #POST请求body 2021-11-28 19:43:38.212 12465 DEBUG cinder.api.v3.volumes [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Create volume request body: {u'volume': {u'backup_id': None, u'description': u'volume1111', u'imageRef': None, u'availability_zone': u'nova', u'source_volid': None, u'name': u'volume1111', u'consistencygroup_id': None, u'volume_type': u'iscsi', u'snapshot_id': None, u'metadata': {}, u'size': 1}} create /usr/lib/python2.7/site-packages/cinder/api/v3/volumes.py:255 #看起来create方法在/usr/lib/python2.7/site-packages/cinder/api/v3/volumes.py:255 2021-11-28 19:43:38.220 12465 INFO cinder.api.v3.volumes [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Create volume of 1 GB #创建volume 2021-11-28 19:43:38.224 12465 DEBUG cinder.volume.api [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Availability zone cache updated, next update will occur around 2021-11-28 12:43:38.224765. list_availability_zones /usr/lib/python2.7/site-packages/cinder/volume/api.py:151 2021-11-28 19:43:38.225 12465 INFO cinder.volume.api [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Availability Zones retrieved successfully. 2021-11-28 19:43:38.240 12465 DEBUG cinder.volume.api [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Flow 'volume_create_api' (8f8b090e-ca1f-44b5-8fdb-ce21310886ee) transitioned into state 'RUNNING' from state 'PENDING' _flow_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:145 #Flow 'volume_create_api'开始；PENDING to RUNNING #volume_create_api 工作流包含若干 Task，每个 Task 完成特定的任务。 这些任务依次为 ExtractVolumeRequestTask, QuotaReserveTask, EntryCreateTask, QuotaCommitTask, VolumeCastTask。 #Task 的执行状态也会经历 PENDING, RUNNING 和 SUCCESS 三个阶段。 2021-11-28 19:43:38.242 12465 DEBUG cinder.volume.api [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Task 'cinder.volume.flows.api.create_volume.ExtractVolumeRequestTask;volume:create' (7983d829-ca13-4502-9c71-e4bcfa6a8fff) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194 2021-11-28 19:43:38.243 12465 DEBUG cinder.volume.flows.api.create_volume [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Validating volume size '1' using validate_int _extract_size /usr/lib/python2.7/site-packages/cinder/volume/flows/api/create_volume.py:206 2021-11-28 19:43:38.275 12465 DEBUG cinder.volume.api [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Task 'cinder.volume.flows.api.create_volume.ExtractVolumeRequestTask;volume:create' (7983d829-ca13-4502-9c71-e4bcfa6a8fff) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'volume_type_id': u'a1f1c9e9-ab39-4989-9837-45113ad19ad2', 'backup_id': None, 'multiattach': False, 'refresh_az': False, 'source_volid': None, 'qos_specs': None, 'consistencygroup_id': None, 'replication_status': 'disabled', 'snapshot_id': None, 'size': 1, 'availability_zones': [u'nova'], 'volume_type': VolumeType(created_at=2021-11-19T14:59:32Z,deleted=False,deleted_at=None,description=None,extra_specs={volume_backend_name='lvm'},id=a1f1c9e9-ab39-4989-9837-45113ad19ad2,is_public=True,name='iscsi',projects=[],qos_specs=<?>,qos_specs_id=None,updated_at=None), 'cgsnapshot_id': None, 'encryption_key_id': None, 'group_id': None}' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183 #ExtractVolumeRequestTask 获取 request 信息 2021-11-28 19:43:38.277 12465 DEBUG cinder.volume.api [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Task 'cinder.volume.flows.api.create_volume.QuotaReserveTask;volume:create' (21d56932-dc75-4e6e-b763-ea19cdd822fc) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194 2021-11-28 19:43:38.324 12465 DEBUG cinder.quota [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Created reservations ['49a9dd9f-5dcf-42fe-96a3-75db859366f0', 'd038ae16-13a5-4c26-b6ff-2cefa8cb1469', '63ab811e-2a55-4d8f-9e86-160ef15e772d', '50543791-21ee-48f4-adc7-4696ae6a2643'] reserve /usr/lib/python2.7/site-packages/cinder/quota.py:1036 2021-11-28 19:43:38.325 12465 DEBUG cinder.volume.api [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Task 'cinder.volume.flows.api.create_volume.QuotaReserveTask;volume:create' (21d56932-dc75-4e6e-b763-ea19cdd822fc) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'reservations': ['49a9dd9f-5dcf-42fe-96a3-75db859366f0', 'd038ae16-13a5-4c26-b6ff-2cefa8cb1469', '63ab811e-2a55-4d8f-9e86-160ef15e772d', '50543791-21ee-48f4-adc7-4696ae6a2643']}' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183 #QuotaReserveTask 预留配额 2021-11-28 19:43:38.326 12465 DEBUG cinder.volume.api [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Task 'cinder.volume.flows.api.create_volume.EntryCreateTask;volume:create' (e1d7d0fe-bb48-4740-8586-c45fb75bfbc6) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194 2021-11-28 19:43:38.358 12465 DEBUG cinder.volume.api [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Task 'cinder.volume.flows.api.create_volume.EntryCreateTask;volume:create' (e1d7d0fe-bb48-4740-8586-c45fb75bfbc6) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'volume': Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2021-11-28T11:43:38Z,deleted=False,deleted_at=None,display_description='volume1111',display_name='volume1111',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host=None,id=07019584-9f04-4a60-82e5-677ee44f9307,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f911df6b22d54d34b716efd7a97edf92',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=None,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=None,user_id='80f373269d7a481b931a9352b42b28b8',volume_attachment=<?>,volume_type=<?>,volume_type_id=a1f1c9e9-ab39-4989-9837-45113ad19ad2), 'volume_properties': VolumeProperties(attach_status='detached',availability_zone='nova',cgsnapshot_id=None,consistencygroup_id=None,display_description='volume1111',display_name='volume1111',encryption_key_id=None,group_id=None,group_type_id=<?>,metadata={},multiattach=False,project_id='f911df6b22d54d34b716efd7a97edf92',qos_specs=None,replication_status=<?>,reservations=['49a9dd9f-5dcf-42fe-96a3-75db859366f0','d038ae16-13a5-4c26-b6ff-2cefa8cb1469','63ab811e-2a55-4d8f-9e86-160ef15e772d','50543791-21ee-48f4-adc7-4696ae6a2643'],size=1,snapshot_id=None,source_replicaid=<?>,source_volid=None,status='creating',user_id='80f373269d7a481b931a9352b42b28b8',volume_type_id=a1f1c9e9-ab39-4989-9837-45113ad19ad2), 'volume_id': '07019584-9f04-4a60-82e5-677ee44f9307'}' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183 #EntryCreateTask 在数据库中创建 volume 条目 2021-11-28 19:43:38.360 12465 DEBUG cinder.volume.api [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Task 'cinder.volume.flows.api.create_volume.QuotaCommitTask;volume:create' (2a4c14ec-e2fd-4377-8c0a-7925b7d2cb2f) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194 2021-11-28 19:43:38.378 12465 DEBUG cinder.volume.api [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Task 'cinder.volume.flows.api.create_volume.QuotaCommitTask;volume:create' (2a4c14ec-e2fd-4377-8c0a-7925b7d2cb2f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'volume_properties': VolumeProperties(attach_status='detached',availability_zone='nova',cgsnapshot_id=None,consistencygroup_id=None,display_description='volume1111',display_name='volume1111',encryption_key_id=None,group_id=None,group_type_id=<?>,metadata={},multiattach=False,project_id='f911df6b22d54d34b716efd7a97edf92',qos_specs=None,replication_status=<?>,reservations=['49a9dd9f-5dcf-42fe-96a3-75db859366f0','d038ae16-13a5-4c26-b6ff-2cefa8cb1469','63ab811e-2a55-4d8f-9e86-160ef15e772d','50543791-21ee-48f4-adc7-4696ae6a2643'],size=1,snapshot_id=None,source_replicaid=<?>,source_volid=None,status='creating',user_id='80f373269d7a481b931a9352b42b28b8',volume_type_id=a1f1c9e9-ab39-4989-9837-45113ad19ad2)}' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183 #QuotaCommitTask 确认配额 2021-11-28 19:43:38.379 12465 DEBUG cinder.volume.api [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Task 'cinder.volume.flows.api.create_volume.VolumeCastTask;volume:create' (9c30fa33-812f-4a90-a2d1-de2ce007dbbe) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194 2021-11-28 19:43:38.390 12465 DEBUG cinder.volume.api [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Task 'cinder.volume.flows.api.create_volume.VolumeCastTask;volume:create' (9c30fa33-812f-4a90-a2d1-de2ce007dbbe) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183 #VolumeCastTask 是向 cinder-sheduler 发送消息，开始调度工作 #VolumeCastTask 会向RabbitMQ发送请求，但是没有相关的日志 #/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py 2021-11-28 19:43:38.392 12465 DEBUG cinder.volume.api [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Flow 'volume_create_api' (8f8b090e-ca1f-44b5-8fdb-ce21310886ee) transitioned into state 'SUCCESS' from state 'RUNNING' _flow_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:145 #Flow 'volume_create_api'结束 ；RUNNING to SUCCESS 2021-11-28 19:43:38.418 12465 INFO cinder.volume.api [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Create volume request issued successfully. #注意：“volume 创建成功”只是指 cinder-api 已经成功处理了 volume create 请求，将消息发给了 cinder-scheduler，但并不意味 volume 在存储节点上已经成功创建 2021-11-28 19:43:38.428 12465 INFO cinder.api.openstack.wsgi [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] http://192.168.1.21:8776/v3/f911df6b22d54d34b716efd7a97edf92/volumes returned with HTTP 202 2021-11-28 19:43:38.429 12465 INFO eventlet.wsgi.server [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] 192.168.1.21 "POST /v3/f911df6b22d54d34b716efd7a97edf92/volumes HTTP/1.1" status: 202 len: 1229 time: 0.2214382 ---------------------------------------------------------------------------------------------------------- #create方法 [root@controller cinder]# cat /usr/lib/python2.7/site-packages/cinder/api/v3/volumes.py def create(self, req, body): """Creates a new volume. :param req: the request :param body: the request body :returns: dict -- the new volume dictionary :raises HTTPNotFound, HTTPBadRequest: """ LOG.debug('Create volume request body: %s', body) context = req.environ['cinder.context'] req_version = req.api_version_request # NOTE (pooja_jadhav) To fix bug 1774155, scheduler hints is not # loaded as a standard extension. If user passes # OS-SCH-HNT:scheduler_hints in the request body, then it will be # validated in the create method and this method will add # scheduler_hints in body['volume']. body = scheduler_hints.create(req, body) volume = body['volume'] kwargs = {} self.validate_name_and_description(volume, check_length=False) # NOTE(thingee): v2 API allows name instead of display_name if 'name' in volume: volume['display_name'] = volume.pop('name') # NOTE(thingee): v2 API allows description instead of # display_description if 'description' in volume: volume['display_description'] = volume.pop('description') if 'image_id' in volume: volume['imageRef'] = volume.pop('image_id') req_volume_type = volume.get('volume_type', None) if req_volume_type: # Not found exception will be handled at the wsgi level kwargs['volume_type'] = ( objects.VolumeType.get_by_name_or_id(context, req_volume_type)) kwargs['metadata'] = volume.get('metadata', None) snapshot_id = volume.get('snapshot_id') if snapshot_id is not None: # Not found exception will be handled at the wsgi level kwargs['snapshot'] = self.volume_api.get_snapshot(context, snapshot_id) else: kwargs['snapshot'] = None source_volid = volume.get('source_volid') if source_volid is not None: # Not found exception will be handled at the wsgi level kwargs['source_volume'] = ( self.volume_api.get_volume(context, source_volid)) else: kwargs['source_volume'] = None kwargs['group'] = None kwargs['consistencygroup'] = None consistencygroup_id = volume.get('consistencygroup_id') if consistencygroup_id is not None: # Not found exception will be handled at the wsgi level kwargs['group'] = self.group_api.get(context, consistencygroup_id) # Get group_id if volume is in a group. group_id = volume.get('group_id') if group_id is not None: # Not found exception will be handled at the wsgi level kwargs['group'] = self.group_api.get(context, group_id) image_ref = volume.get('imageRef') if image_ref is not None: image_uuid = self._image_uuid_from_ref(image_ref, context) image_snapshot = self._get_image_snapshot(context, image_uuid) if (req_version.matches(mv.get_api_version( mv.SUPPORT_NOVA_IMAGE)) and image_snapshot): kwargs['snapshot'] = image_snapshot else: kwargs['image_id'] = image_uuid backup_id = volume.get('backup_id') if backup_id: kwargs['backup'] = self.backup_api.get(context, backup_id=backup_id) size = volume.get('size', None) if size is None and kwargs['snapshot'] is not None: size = kwargs['snapshot']['volume_size'] elif size is None and kwargs['source_volume'] is not None: size = kwargs['source_volume']['size'] elif size is None and kwargs.get('backup') is not None: size = kwargs['backup']['size'] LOG.info("Create volume of %s GB", size) kwargs['availability_zone'] = volume.get('availability_zone', None) kwargs['scheduler_hints'] = volume.get('scheduler_hints', None) multiattach = volume.get('multiattach', False) kwargs['multiattach'] = multiattach if multiattach: msg = ("The option 'multiattach' " "is deprecated and will be removed in a future " "release. The default behavior going forward will " "be to specify multiattach enabled volume types.") versionutils.report_deprecated_feature(LOG, msg) try: new_volume = self.volume_api.create( context, size, volume.get('display_name'), volume.get('display_description'), **kwargs) except exception.VolumeTypeDefaultMisconfiguredError as err: raise exc.HTTPInternalServerError(explanation=err.msg) retval = self._view_builder.detail(req, new_volume) return retval #这种api不知道在哪个文件的话，那么只能是遍历所有文件内容，然后进行过滤了 cat /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py def _reschedule(self, context, cause, request_spec, filter_properties, volume): """Actions that happen during the rescheduling attempt occur here.""" create_volume = self.scheduler_rpcapi.create_volume #self.scheduler_rpcapi.create_volume就是向volume rabbitMQ队列发送create_volume方法 if not filter_properties: filter_properties = {} if 'retry' not in filter_properties: filter_properties['retry'] = {} retry_info = filter_properties['retry'] num_attempts = retry_info.get('num_attempts', 0) request_spec['volume_id'] = volume.id LOG.debug("Volume %(volume_id)s: re-scheduling %(method)s " "attempt %(num)d due to %(reason)s", {'volume_id': volume.id, 'method': common.make_pretty_name(create_volume), 'num': num_attempts, 'reason': cause.exception_str}) if all(cause.exc_info): # Stringify to avoid circular ref problem in json serialization retry_info['exc'] = traceback.format_exception(*cause.exc_info) return create_volume(context, volume, request_spec=request_spec, filter_properties=filter_properties) #这里才是真正地向volume rabbitMQ队列发送create_volume方法
2.cinder-scheduler 执行调度算法，向volume队列发送create_volume任务 =================================================================================================================== cinder-scheduler 执行调度算法，通过 Filter 和 Weigher 挑选最优的存储节点 [root@controller cinder]# cat /var/log/cinder/scheduler.log |grep "req-e419e8b4-b40f-45af-8777-2330e8600512" 2021-11-28 19:43:38.408 12067 DEBUG cinder.scheduler.manager [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Flow 'volume_create_scheduler' (4b3bb121-cdbc-4603-b688-9db869d35e4d) transitioned into state 'RUNNING' from state 'PENDING' _flow_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:145 #Flow 'volume_create_scheduler'流开始 #Flow 依次执行 ExtractSchedulerSpecTask 和 ScheduleCreateVolumeTask 2021-11-28 19:43:38.411 12067 DEBUG cinder.scheduler.manager [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Task 'cinder.scheduler.flows.create_volume.ExtractSchedulerSpecTask;volume:create' (216e6373-bc50-4f77-9411-176b060d7d52) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194 #ExtractSchedulerSpecTask 开始 2021-11-28 19:43:38.412 12067 DEBUG cinder.scheduler.manager [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Task 'cinder.scheduler.flows.create_volume.ExtractSchedulerSpecTask;volume:create' (216e6373-bc50-4f77-9411-176b060d7d52) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'request_spec': RequestSpec(CG_backend=<?>,availability_zones=['nova'],backup_id=None,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=None,operation='create_volume',resource_backend=<?>,snapshot_id=None,source_replicaid=<?>,source_volid=None,volume=Volume(07019584-9f04-4a60-82e5-677ee44f9307),volume_id=07019584-9f04-4a60-82e5-677ee44f9307,volume_properties=VolumeProperties,volume_type=VolumeType(a1f1c9e9-ab39-4989-9837-45113ad19ad2))}' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183 #ExtractSchedulerSpecTask 完成 2021-11-28 19:43:38.414 12067 DEBUG cinder.scheduler.manager [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Task 'cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create' (680be0f3-7b1e-4c08-9e6c-67b177d08af3) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194 #ScheduleCreateVolumeTask 开始 #主要的 filter 和 weighting 工作由 ScheduleCreateVolumeTask 完成 2021-11-28 19:43:38.420 12067 DEBUG oslo_db.sqlalchemy.engines [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:307 2021-11-28 19:43:38.440 12067 DEBUG cinder.scheduler.host_manager [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Updating capabilities for controller@lvm#lvm: {u'pool_name': u'lvm', u'filter_function': None, u'goodness_function': None, u'multiattach': True, u'total_volumes': 3, u'provisioned_capacity_gb': 2.0, 'timestamp': datetime.datetime(2021, 11, 28, 11, 43, 6, 667575), u'allocated_capacity_gb': 2, 'volume_backend_name': u'lvm', u'thin_provisioning_support': True, u'free_capacity_gb': 19.46, 'driver_version': u'3.0.0', u'location_info': u'LVMVolumeDriver:controller:cinder-volumes:thin:0', u'total_capacity_gb': 19.57, u'thick_provisioning_support': False, u'reserved_percentage': 0, u'QoS_support': False, u'max_over_subscription_ratio': u'20.0', 'vendor_name': u'Open Source', 'storage_protocol': u'iSCSI', u'backend_state': u'up'} update_from_volume_capability /usr/lib/python2.7/site-packages/cinder/scheduler/host_manager.py:361 2021-11-28 19:43:38.440 12067 DEBUG cinder.scheduler.base_filter [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Starting with 1 host(s) get_filtered_objects /usr/lib/python2.7/site-packages/cinder/scheduler/base_filter.py:95 #开始过滤 2021-11-28 19:43:38.441 12067 DEBUG cinder.scheduler.base_filter [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Filter AvailabilityZoneFilter returned 1 host(s) get_filtered_objects /usr/lib/python2.7/site-packages/cinder/scheduler/base_filter.py:125 2021-11-28 19:43:38.441 12067 DEBUG cinder.scheduler.filters.capacity_filter [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Checking if host controller@lvm#lvm can create a 1 GB volume (07019584-9f04-4a60-82e5-677ee44f9307) backend_passes /usr/lib/python2.7/site-packages/cinder/scheduler/filters/capacity_filter.py:62 2021-11-28 19:43:38.441 12067 DEBUG cinder.scheduler.filters.capacity_filter [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Checking provisioning for request of 1 GB. Backend: host 'controller@lvm#lvm':free_capacity_gb: 19.46, total_capacity_gb: 19.57,allocated_capacity_gb: 2, max_over_subscription_ratio: 20.0,reserved_percentage: 0, provisioned_capacity_gb: 2.0,thin_provisioning_support: True, thick_provisioning_support: False,pools: None,updated at: 2021-11-28 11:43:06.667575 backend_passes /usr/lib/python2.7/site-packages/cinder/scheduler/filters/capacity_filter.py:134 2021-11-28 19:43:38.442 12067 DEBUG cinder.scheduler.filters.capacity_filter [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Space information for volume creation on host controller@lvm#lvm (requested / avail): 1/19.46 backend_passes /usr/lib/python2.7/site-packages/cinder/scheduler/filters/capacity_filter.py:172 2021-11-28 19:43:38.442 12067 DEBUG cinder.scheduler.base_filter [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Filter CapacityFilter returned 1 host(s) get_filtered_objects /usr/lib/python2.7/site-packages/cinder/scheduler/base_filter.py:125 2021-11-28 19:43:38.442 12067 DEBUG cinder.scheduler.base_filter [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Filter CapabilitiesFilter returned 1 host(s) get_filtered_objects /usr/lib/python2.7/site-packages/cinder/scheduler/base_filter.py:125 2021-11-28 19:43:38.442 12067 DEBUG cinder.scheduler.filter_scheduler [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Filtered [host 'controller@lvm#lvm':free_capacity_gb: 19.46, total_capacity_gb: 19.57,allocated_capacity_gb: 2, max_over_subscription_ratio: 20.0,reserved_percentage: 0, provisioned_capacity_gb: 2.0,thin_provisioning_support: True, thick_provisioning_support: False,pools: None,updated at: 2021-11-28 11:43:06.667575] _get_weighted_candidates /usr/lib/python2.7/site-packages/cinder/scheduler/filter_scheduler.py:348 #这个看起来是过滤结果？ 2021-11-28 19:43:38.443 12067 DEBUG cinder.scheduler.base_weight [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Weigher CapacityWeigher returned, weigher value is {max: 389.4, min: 389.4} get_weighed_objects /usr/lib/python2.7/site-packages/cinder/scheduler/base_weight.py:153 2021-11-28 19:43:38.443 12067 DEBUG cinder.scheduler.host_manager [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Weighed [WeighedHost [host: controller@lvm#lvm, weight: 0.0]] get_weighed_backends /usr/lib/python2.7/site-packages/cinder/scheduler/host_manager.py:503 #Weigher结果 2021-11-28 19:43:38.443 12067 DEBUG cinder.scheduler.filter_scheduler [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Choosing controller@lvm#lvm _choose_top_backend /usr/lib/python2.7/site-packages/cinder/scheduler/filter_scheduler.py:569 #选择了controller@lvm#lvm _choose_top_backend 2021-11-28 19:43:38.443 12067 DEBUG cinder.scheduler.host_manager [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Consumed 1 GB from backend: host 'controller@lvm#lvm':free_capacity_gb: 18.46, total_capacity_gb: 19.57,allocated_capacity_gb: 3, max_over_subscription_ratio: 20.0,reserved_percentage: 0, provisioned_capacity_gb: 3.0,thin_provisioning_support: True, thick_provisioning_support: False,pools: None,updated at: 2021-11-28 11:43:38.443930 consume_from_volume /usr/lib/python2.7/site-packages/cinder/scheduler/host_manager.py:319 2021-11-28 19:43:38.481 12067 DEBUG cinder.scheduler.manager [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Task 'cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create' (680be0f3-7b1e-4c08-9e6c-67b177d08af3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183 #ScheduleCreateVolumeTask 完成 2021-11-28 19:43:38.482 12067 DEBUG cinder.scheduler.manager [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Flow 'volume_create_scheduler' (4b3bb121-cdbc-4603-b688-9db869d35e4d) transitioned into state 'SUCCESS' from state 'RUNNING' _flow_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:145 #Flow 'volume_create_scheduler' 流结束 ---------------------------------------------------------------------------- #scheduler调度向rabbitMQ发送任务的api cat /usr/lib/python2.7/site-packages/cinder/scheduler/filter_scheduler.py def schedule_create_volume(self, context, request_spec, filter_properties):##虽然这个api没有相关的log，但是相应的filter_scheduler.py文件中的方法其实被多次使用了 backend = self._schedule(context, request_spec, filter_properties) if not backend: raise exception.NoValidBackend(reason=_("No weighed backends " "available")) backend = backend.obj volume_id = request_spec['volume_id'] updated_volume = driver.volume_update_db( context, volume_id, backend.host, backend.cluster_name, availability_zone=backend.service['availability_zone']) self._post_select_populate_filter_properties(filter_properties, backend) # context is not serializable filter_properties.pop('context', None) self.volume_rpcapi.create_volume(context, updated_volume, request_spec,filter_properties,allow_reschedule=True) #向volume队列发送create_volume任务
3.cinder-volume 的处理过程 [root@controller cinder]# cat /var/log/cinder/volume.log |grep req-e419e8b4-b40f-45af-8777-2330e8600512 2021-11-28 19:43:38.506 12515 DEBUG cinder.volume.flows.manager.create_volume [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Volume reschedule parameters: True retry: {u'num_attempts': 1, u'backends': [u'controller@lvm#lvm'], u'hosts': [u'controller@lvm#lvm']} get_flow /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:1265 2021-11-28 19:43:38.522 12515 DEBUG cinder.volume.manager [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Flow 'volume_create_manager' (52531783-7bcd-4851-916a-6205993ad094) transitioned into state 'RUNNING' from state 'PENDING' _flow_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:145 #Flow 'volume_create_manager' 流开始 #1.volume_create_manager 首先执行 ExtractVolumeRefTask, OnFailureRescheduleTask, ExtractVolumeSpecTask, NotifyVolumeActionTask 为 volume 创建做准备。 2021-11-28 19:43:38.525 12515 DEBUG cinder.volume.manager [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3c875a60-91d9-4990-bb5f-be95f6532baf) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194 2021-11-28 19:43:38.541 12515 DEBUG oslo_db.sqlalchemy.engines [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:307 2021-11-28 19:43:38.560 12515 DEBUG cinder.volume.manager [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3c875a60-91d9-4990-bb5f-be95f6532baf) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2021-11-28T11:43:38Z,deleted=False,deleted_at=None,display_description='volume1111',display_name='volume1111',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='controller@lvm#lvm',id=07019584-9f04-4a60-82e5-677ee44f9307,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f911df6b22d54d34b716efd7a97edf92',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2021-11-28T11:43:38Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2021-11-28T11:43:38Z,user_id='80f373269d7a481b931a9352b42b28b8',volume_attachment=<?>,volume_type=VolumeType(a1f1c9e9-ab39-4989-9837-45113ad19ad2),volume_type_id=a1f1c9e9-ab39-4989-9837-45113ad19ad2)' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183 2021-11-28 19:43:38.563 12515 DEBUG cinder.volume.manager [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3b0b707d-d850-43fa-9ef0-ab8862d6e7a2) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194 2021-11-28 19:43:38.565 12515 DEBUG cinder.volume.manager [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3b0b707d-d850-43fa-9ef0-ab8862d6e7a2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183 2021-11-28 19:43:38.566 12515 DEBUG cinder.volume.manager [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (34705fb8-ed25-435a-9556-c53fafc1b561) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194 2021-11-28 19:43:38.568 12515 DEBUG cinder.volume.manager [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (34705fb8-ed25-435a-9556-c53fafc1b561) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': u'creating', 'volume_size': 1, 'volume_name': u'volume-07019584-9f04-4a60-82e5-677ee44f9307', 'type': 'raw', 'volume_id': u'07019584-9f04-4a60-82e5-677ee44f9307'}' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183 2021-11-28 19:43:38.570 12515 DEBUG cinder.volume.manager [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c839d77c-eb59-45ff-8742-875df495cf3d) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194 2021-11-28 19:43:38.592 12515 DEBUG cinder.volume.manager [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c839d77c-eb59-45ff-8742-875df495cf3d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183 2021-11-28 19:43:38.593 12515 DEBUG cinder.volume.manager [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8a75786c-78fb-4bbf-9a43-5a4e864cac8f) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194 #2.接下来 CreateVolumeFromSpecTask 执行 volume 创建任务。 2021-11-28 19:43:38.594 12515 INFO cinder.volume.flows.manager.create_volume [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Volume 07019584-9f04-4a60-82e5-677ee44f9307: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': u'volume-07019584-9f04-4a60-82e5-677ee44f9307'} #{'status': u'creating', 'volume_size': 1, 'volume_name': u'volume-07019584-9f04-4a60-82e5-677ee44f9307'} 2021-11-28 19:43:38.595 12515 DEBUG oslo_concurrency.processutils [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C lvcreate -T -V 1g -n volume-07019584-9f04-4a60-82e5-677ee44f9307 cinder-volumes/cinder-volumes-pool execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:379 2021-11-28 19:43:38.946 12515 DEBUG oslo_concurrency.processutils [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C lvcreate -T -V 1g -n volume-07019584-9f04-4a60-82e5-677ee44f9307 cinder-volumes/cinder-volumes-pool" returned: 0 in 0.351s execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:416 2021-11-28 19:43:38.949 12515 DEBUG cinder.volume.manager [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8a75786c-78fb-4bbf-9a43-5a4e864cac8f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': u'creating', 'volume_size': 1, 'volume_name': u'volume-07019584-9f04-4a60-82e5-677ee44f9307'}' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183 2021-11-28 19:43:38.951 12515 DEBUG cinder.volume.manager [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (876df64e-eb38-4f45-9842-a1d7fa0d2a56) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194 #3.CreateVolumeOnFinishTask 完成扫尾工作。 2021-11-28 19:43:38.993 12515 INFO cinder.volume.flows.manager.create_volume [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Volume volume-07019584-9f04-4a60-82e5-677ee44f9307 (07019584-9f04-4a60-82e5-677ee44f9307): created successfully 2021-11-28 19:43:38.994 12515 DEBUG cinder.volume.manager [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (876df64e-eb38-4f45-9842-a1d7fa0d2a56) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183 2021-11-28 19:43:38.996 12515 DEBUG cinder.volume.manager [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Flow 'volume_create_manager' (52531783-7bcd-4851-916a-6205993ad094) transitioned into state 'SUCCESS' from state 'RUNNING' _flow_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:145 #Flow 'volume_create_manager' 流结束 2021-11-28 19:43:39.004 12515 INFO cinder.volume.manager [req-e419e8b4-b40f-45af-8777-2330e8600512 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Created volume successfully. [root@controller ~(keystone_admin)]# cinder list +--------------------------------------+-----------+------------+------+-------------+----------+--------------------------------------+ | ID | Status | Name | Size | Volume Type | Bootable | Attached to | +--------------------------------------+-----------+------------+------+-------------+----------+--------------------------------------+ | 07019584-9f04-4a60-82e5-677ee44f9307 | available | volume1111 | 1 | iscsi | false | | | 3660c6ed-0e88-4d12-a9ba-d9653faa99f2 | in-use | | 1 | iscsi | true | dfd40b8f-6886-44d2-a839-0056c5da6b79 | | 8f9b3474-469f-4875-9ae3-db0330cd4e0d | error | | 1 | iscsi | false | | +--------------------------------------+-----------+------------+------+-------------+----------+--------------------------------------+
在cinder中，一个任务触发1个flow，1个flow中包含了多个task
但是在nova中并非如此，尝试了launch实例，nova-api.log在没有找到flow和 task
---------------------------------------------------------------------------------------------
Volume 的最主要用途是作为虚拟硬盘提供给 instance 使用。Volume 是通过 Attach 操作挂载到 instance 上的。
通常情况存储节点和计算节点是不同的物理节点。解决方案是使用 iSCSI
iSCSI 是 Client-Server 架构
Target：提供 iSCSI 存储资源的设备，简单的说，就是 iSCSI 服务器。
Initiator：使用 iSCSI 存储资源的设备，也就是 iSCSI 客户端。
Initiator 需要与 target 建立 iSCSI 连接，执行 login 操作，然后就可以使用 target 上面的块存储设备了。 Target 提供的块存储设备支持多种实现方式
Cinder 的存储节点 cinder-volume 默认使用 tgt 软件来管理和监控 iSCSI target，在计算节点 nova-compute 使用 iscsiadm 执行 initiator 相关操作。
Volume Attach 操作的流程；日志 ==================================================================================== 白屏-->Volumes-->Volumes-->选择 volume ，点击“Manage Attachments”-->Attach to Instance Volume Attach 操作的流程: 1.向 cinder-api 发送 attach 请求 2.cinder-api 发送消息 attach操作需要cinder-volume和nova-compute2个组件进行协作（个人理解因为2个组件是处于2个不同的服务模块下，所以需要api向rabbitMQ发送两条信息，分别进行处理；而不是类似launch instance，因为是同一个服务模块内部，采用的是迭代协作） 3.cinder-volume 初始化 volume 的连接 在Volume 创建后，只是在 volume provider 中创建了相应存储对象（比如 LV），这时计算节点是无法使用的。 初始化 volume 的连接（export 过程）：Cinder-volume 需要以某种方式将 volume export 出来，计算节点才能够访问得到。 cinder-api 的日志文件 /opt/stack/logs/c-api.log 中记录的相关信息 Initialize_connection 的具体工作主要由 cinder-volume 完成的 4.nova-compute 将 volume attach 到 instance 计算节点将 volume 挂载到指定的 instance，完成 attach 操作 cinder-api 的日志文件 /opt/stack/logs/c-api.log Attach 的具体工作主要由 nova-compute 完成 --------------------------------------------------------------------------------------------------------------------------------------------------- Volume Attach 操作的流程: 1.向 cinder-api 发送 attach 请求 2.cinder-api 发送消息 attach操作需要cinder-volume和nova-compute2个组件进行协作（个人理解因为2个组件是处于2个不同的服务模块下，所以需要api向rabbitMQ发送两条信息，分别进行处理；而不是类似launch instance，因为是同一个服务模块内部，采用的是迭代协作） ###cinder-api向rabbitMQ发送initialize_connection请求： #/var/log/cinder/api.log #此日志是通过/var/log/cinder/volume.log的日志获取requestID，倒推出来的 [root@controller ~]# cat /var/log/cinder/api.log |grep -i req-a69412f0-fdff-46e2-a340-28d3d44a6089 2021-12-05 07:05:02.035 12467 INFO cinder.api.openstack.wsgi [req-a69412f0-fdff-46e2-a340-28d3d44a6089 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] PUT http://192.168.1.21:8776/v3/f911df6b22d54d34b716efd7a97edf92/attachments/0bbb9853-94c9-4165-930b-18e59e53bf27 #put方法？launch实例使用的post方法 2021-12-05 07:05:02.035 12467 DEBUG cinder.api.openstack.wsgi [req-a69412f0-fdff-46e2-a340-28d3d44a6089 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Action: 'update', calling method: version_select, body: {"attachment": {"connector": {"initiator": "iqn.1994-05.com.redhat:51f266169dc3", "ip": "192.168.1.22", "system uuid": "B2E84D56-40A1-3B4E-11F3-62D43C5756EC", "platform": "x86_64", "host": "compute22", "do_local_attach": false, "mountpoint": "/dev/vdb", "os_type": "linux2", "multipath": false}}} _process_stack /usr/lib/python2.7/site-packages/cinder/api/openstack/wsgi.py:876 2021-12-05 07:05:02.049 12467 DEBUG cinder.coordination [req-a69412f0-fdff-46e2-a340-28d3d44a6089 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Lock "/var/lib/cinder/cinder-attachment_update-b887f3ec-f5ac-4a3c-a729-9f4bba097a55-compute22" acquired by "attachment_update" :: waited 0.000s _synchronized /usr/lib/python2.7/site-packages/cinder/coordination.py:150 2021-12-05 07:05:04.621 12467 DEBUG cinder.coordination [req-a69412f0-fdff-46e2-a340-28d3d44a6089 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Lock "/var/lib/cinder/cinder-attachment_update-b887f3ec-f5ac-4a3c-a729-9f4bba097a55-compute22" released by "attachment_update" :: held 2.572s _synchronized /usr/lib/python2.7/site-packages/cinder/coordination.py:162 2021-12-05 07:05:04.622 12467 INFO cinder.api.openstack.wsgi [req-a69412f0-fdff-46e2-a340-28d3d44a6089 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] http://192.168.1.21:8776/v3/f911df6b22d54d34b716efd7a97edf92/attachments/0bbb9853-94c9-4165-930b-18e59e53bf27 returned with HTTP 200 2021-12-05 07:05:04.622 12467 INFO eventlet.wsgi.server [req-a69412f0-fdff-46e2-a340-28d3d44a6089 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] 192.168.1.22 "PUT /v3/f911df6b22d54d34b716efd7a97edf92/attachments/0bbb9853-94c9-4165-930b-18e59e53bf27 HTTP/1.1" status: 200 len: 1071 time: 2.5900531 #initialize_connection请求api: #在launch instance时，使用这个方法，也能在/var/log/cinder/api.log看到相应的successfully日志 #虽然教程说使用的是这个方法，但是我存疑,因为在Volume Attach时，不能找到相应的successfully日志 def initialize_connection(self, context, volume, connector): context.authorize(vol_action_policy.INITIALIZE_POLICY, target_obj=volume) if volume.status == 'maintenance': LOG.info('Unable to initialize the connection for ' 'volume, because it is in ' 'maintenance.', resource=volume) msg = _("The volume connection cannot be initialized in " "maintenance mode.") raise exception.InvalidVolume(reason=msg) init_results = self.volume_rpcapi.initialize_connection(context,volume,connector) LOG.info("Initialize volume connection completed successfully.",resource=volume) return init_results -------------------- ###nova-compute向rabbitMQ发送发送attach_volume请求 #日志/var/log/nova/nova-api.log [root@controller ~]# cat /var/log/nova/nova-api.log |grep -i req-0d6fd23c-0636-46e0-9866-452713734f80 2021-12-05 07:05:00.697 15854 DEBUG nova.api.openstack.wsgi [req-0d6fd23c-0636-46e0-9866-452713734f80 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Action: 'create', calling method: <bound method VolumeAttachmentController.create of <nova.api.openstack.compute.volumes.VolumeAttachmentController object at 0x7fe64280d950>>, body: {"volumeAttachment": {"volumeId": "b887f3ec-f5ac-4a3c-a729-9f4bba097a55"}} _process_stack /usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py:520 #根据volumeId判断，确实是这个 requestID #这个requestID无法在其他log文件中找到日志 2021-12-05 07:05:00.699 15854 DEBUG nova.compute.api [req-0d6fd23c-0636-46e0-9866-452713734f80 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] [instance: fd513532-4a76-480f-a64d-a1e842901296] Fetching instance by UUID get /usr/lib/python2.7/site-packages/nova/compute/api.py:2654 2021-12-05 07:05:00.704 15854 DEBUG oslo_concurrency.lockutils [req-0d6fd23c-0636-46e0-9866-452713734f80 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Lock "4809eaf7-095b-41d7-a897-4a885c87aca9" acquired by "nova.context.get_or_set_cached_cell_and_set_connections" :: waited 0.000s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:327 2021-12-05 07:05:00.705 15854 DEBUG oslo_concurrency.lockutils [req-0d6fd23c-0636-46e0-9866-452713734f80 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Lock "4809eaf7-095b-41d7-a897-4a885c87aca9" released by "nova.context.get_or_set_cached_cell_and_set_connections" :: held 0.000s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:339 2021-12-05 07:05:00.705 15854 DEBUG oslo_concurrency.lockutils [req-0d6fd23c-0636-46e0-9866-452713734f80 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Lock "4809eaf7-095b-41d7-a897-4a885c87aca9" acquired by "nova.context.get_or_set_cached_cell_and_set_connections" :: waited 0.000s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:327 2021-12-05 07:05:00.705 15854 DEBUG oslo_concurrency.lockutils [req-0d6fd23c-0636-46e0-9866-452713734f80 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Lock "4809eaf7-095b-41d7-a897-4a885c87aca9" released by "nova.context.get_or_set_cached_cell_and_set_connections" :: held 0.000s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:339 2021-12-05 07:05:01.616 15854 INFO nova.api.openstack.requestlog [req-0d6fd23c-0636-46e0-9866-452713734f80 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] 192.168.1.21 "POST /v2.1/f911df6b22d54d34b716efd7a97edf92/servers/fd513532-4a76-480f-a64d-a1e842901296/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.921668 #attach_volume请求API： #nova-compute完成attach操作，但是调用的却是self.volume_rpcapi.attach_volume()方法？跨服务模块调用？ #怀疑attach_volume请求同样不是使用这个api，因为没有找到相关的"Attach volume completed successfully."日志；但是在launch实例时，可以看到该日志 def attach(self, context, volume, instance_uuid, host_name, mountpoint, mode): context.authorize(vol_action_policy.ATTACH_POLICY, target_obj=volume) if volume.status == 'maintenance': LOG.info('Unable to attach volume, ' 'because it is in maintenance.', resource=volume) msg = _("The volume cannot be attached in maintenance mode.") raise exception.InvalidVolume(reason=msg) # We add readonly metadata if it doesn't already exist readonly = self.update_volume_admin_metadata(context.elevated(), volume, {'readonly': 'False'}, update=False)['readonly'] if readonly == 'True' and mode != 'ro': raise exception.InvalidVolumeAttachMode(mode=mode, volume_id=volume.id) attach_results = self.volume_rpcapi.attach_volume(context,volume,instance_uuid, host_name,mountpoint,mode) LOG.info("Attach volume completed successfully.",resource=volume) return attach_results 3.cinder-volume 初始化 volume 的连接 #通过"Creating iscsi_target for volume"日志锁定了cinder-volume初始化连接的requestID [root@controller ~]# cat /var/log/cinder/volume.log |grep "Creating iscsi_target for volume" 2021-12-05 07:05:02.430 12515 INFO cinder.volume.targets.lio [req-a69412f0-fdff-46e2-a340-28d3d44a6089 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Creating iscsi_target for volume: volume-b887f3ec-f5ac-4a3c-a729-9f4bba097a55 [root@controller ~]# cat /var/log/cinder/volume.log |grep "req-a69412f0-fdff-46e2-a340-28d3d44a6089" 2021-12-05 07:05:02.121 12515 DEBUG oslo_concurrency.processutils [req-a69412f0-fdff-46e2-a340-28d3d44a6089 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf lvchange -a y --yes -K cinder-volumes/volume-b887f3ec-f5ac-4a3c-a729-9f4bba097a55 execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:379 2021-12-05 07:05:02.377 12515 DEBUG oslo_concurrency.processutils [req-a69412f0-fdff-46e2-a340-28d3d44a6089 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf lvchange -a y --yes -K cinder-volumes/volume-b887f3ec-f5ac-4a3c-a729-9f4bba097a55" returned: 0 in 0.256s execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:416 2021-12-05 07:05:02.430 12515 INFO cinder.volume.targets.lio [req-a69412f0-fdff-46e2-a340-28d3d44a6089 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Creating iscsi_target for volume: volume-b887f3ec-f5ac-4a3c-a729-9f4bba097a55 2021-12-05 07:05:02.432 12515 DEBUG oslo_concurrency.lockutils [req-a69412f0-fdff-46e2-a340-28d3d44a6089 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Lock "lioadm" acquired by "cinder.volume.targets.lio._execute" :: waited 0.000s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:327 2021-12-05 07:05:02.433 12515 DEBUG oslo_concurrency.processutils [req-a69412f0-fdff-46e2-a340-28d3d44a6089 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf cinder-rtstool create /dev/cinder-volumes/volume-b887f3ec-f5ac-4a3c-a729-9f4bba097a55 iqn.2010-10.org.openstack:volume-b887f3ec-f5ac-4a3c-a729-9f4bba097a55 Fjc37dhW22oSbsXDRq4K T7PQK469bKshoXCr False -p3260 -a192.168.1.21 execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:379 2021-12-05 07:05:02.849 12515 DEBUG oslo_concurrency.processutils [req-a69412f0-fdff-46e2-a340-28d3d44a6089 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf cinder-rtstool create /dev/cinder-volumes/volume-b887f3ec-f5ac-4a3c-a729-9f4bba097a55 iqn.2010-10.org.openstack:volume-b887f3ec-f5ac-4a3c-a729-9f4bba097a55 Fjc37dhW22oSbsXDRq4K T7PQK469bKshoXCr False -p3260 -a192.168.1.21" returned: 0 in 0.416s execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:416 2021-12-05 07:05:02.850 12515 DEBUG oslo_concurrency.lockutils [req-a69412f0-fdff-46e2-a340-28d3d44a6089 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Lock "lioadm" released by "cinder.volume.targets.lio._execute" :: held 0.418s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:339 2021-12-05 07:05:02.851 12515 DEBUG oslo_concurrency.lockutils [req-a69412f0-fdff-46e2-a340-28d3d44a6089 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Lock "lioadm" acquired by "cinder.volume.targets.lio._execute" :: waited 0.000s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:327 2021-12-05 07:05:02.851 12515 DEBUG oslo_concurrency.processutils [req-a69412f0-fdff-46e2-a340-28d3d44a6089 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf cinder-rtstool get-targets execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:379 2021-12-05 07:05:03.240 12515 DEBUG oslo_concurrency.processutils [req-a69412f0-fdff-46e2-a340-28d3d44a6089 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf cinder-rtstool get-targets" returned: 0 in 0.389s execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:416 2021-12-05 07:05:03.241 12515 DEBUG oslo_concurrency.lockutils [req-a69412f0-fdff-46e2-a340-28d3d44a6089 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Lock "lioadm" released by "cinder.volume.targets.lio._execute" :: held 0.390s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:339 2021-12-05 07:05:03.241 12515 DEBUG oslo_concurrency.lockutils [req-a69412f0-fdff-46e2-a340-28d3d44a6089 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Lock "lioadm" acquired by "cinder.volume.targets.lio._execute" :: waited 0.000s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:327 2021-12-05 07:05:03.242 12515 DEBUG oslo_concurrency.processutils [req-a69412f0-fdff-46e2-a340-28d3d44a6089 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf cinder-rtstool save execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:379 2021-12-05 07:05:03.668 12515 DEBUG oslo_concurrency.processutils [req-a69412f0-fdff-46e2-a340-28d3d44a6089 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf cinder-rtstool save" returned: 0 in 0.427s execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:416 2021-12-05 07:05:03.669 12515 DEBUG oslo_concurrency.lockutils [req-a69412f0-fdff-46e2-a340-28d3d44a6089 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Lock "lioadm" released by "cinder.volume.targets.lio._execute" :: held 0.428s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:339 2021-12-05 07:05:03.670 12515 DEBUG cinder.volume.targets.iscsi [req-a69412f0-fdff-46e2-a340-28d3d44a6089 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Set provider_location to: 192.168.1.21:3260,iqn.2010-10.org.openstack:volume-b887f3ec-f5ac-4a3c-a729-9f4bba097a55 iqn.2010-10.org.openstack:volume-b887f3ec-f5ac-4a3c-a729-9f4bba097a55 0 create_export /usr/lib/python2.7/site-packages/cinder/volume/targets/iscsi.py:213 2021-12-05 07:05:03.678 12515 DEBUG oslo_concurrency.lockutils [req-a69412f0-fdff-46e2-a340-28d3d44a6089 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Lock "lioadm" acquired by "cinder.volume.targets.lio._execute" :: waited 0.000s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:327 2021-12-05 07:05:03.680 12515 DEBUG oslo_concurrency.processutils [req-a69412f0-fdff-46e2-a340-28d3d44a6089 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf cinder-rtstool add-initiator iqn.2010-10.org.openstack:volume-b887f3ec-f5ac-4a3c-a729-9f4bba097a55 Fjc37dhW22oSbsXDRq4K T7PQK469bKshoXCr iqn.1994-05.com.redhat:51f266169dc3 execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:379 2021-12-05 07:05:04.096 12515 DEBUG oslo_concurrency.processutils [req-a69412f0-fdff-46e2-a340-28d3d44a6089 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf cinder-rtstool add-initiator iqn.2010-10.org.openstack:volume-b887f3ec-f5ac-4a3c-a729-9f4bba097a55 Fjc37dhW22oSbsXDRq4K T7PQK469bKshoXCr iqn.1994-05.com.redhat:51f266169dc3" returned: 0 in 0.417s execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:416 2021-12-05 07:05:04.097 12515 DEBUG oslo_concurrency.lockutils [req-a69412f0-fdff-46e2-a340-28d3d44a6089 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Lock "lioadm" released by "cinder.volume.targets.lio._execute" :: held 0.419s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:339 2021-12-05 07:05:04.098 12515 DEBUG oslo_concurrency.lockutils [req-a69412f0-fdff-46e2-a340-28d3d44a6089 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Lock "lioadm" acquired by "cinder.volume.targets.lio._execute" :: waited 0.000s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:327 2021-12-05 07:05:04.099 12515 DEBUG oslo_concurrency.processutils [req-a69412f0-fdff-46e2-a340-28d3d44a6089 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf cinder-rtstool save execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:379 2021-12-05 07:05:04.514 12515 DEBUG oslo_concurrency.processutils [req-a69412f0-fdff-46e2-a340-28d3d44a6089 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf cinder-rtstool save" returned: 0 in 0.415s execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:416 2021-12-05 07:05:04.515 12515 DEBUG oslo_concurrency.lockutils [req-a69412f0-fdff-46e2-a340-28d3d44a6089 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Lock "lioadm" released by "cinder.volume.targets.lio._execute" :: held 0.417s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:339 2021-12-05 07:05:04.609 12515 INFO cinder.volume.manager [req-a69412f0-fdff-46e2-a340-28d3d44a6089 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] attachment_update completed successfully. 4.nova-compute 将 volume attach 到 instance for i in $( find / -path "/proc" -prune -o -name '*\.log*' );do echo "##############$i" >> /tmp/find1205.txt; cat $i |grep -iP 'new|update|login|rescan' >> /tmp/find1205.txt;done #搜索了所有日志文件，结果却没有任何关于'new|update|login|rescan'的日志 #看来相关代码大变样了。。。
launch实例时，cinder-api的Initialize volume日志、Attach volume日志 =================================================================================================== #launch实例可以查到日志，但是attach volume无此日志 [root@controller ~]# cat /var/log/cinder/api.log |grep "Initialize volume connection completed successfully." #launch实例可以查到日志，但是attach volume无此日志 2021-12-05 07:03:51.030 12467 INFO cinder.volume.api [req-71add0be-4246-4a76-8c2c-dbc2f72ec7c4 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Initialize volume connection completed successfully. [root@controller ~]# cat /var/log/cinder/api.log |grep "Attach volume completed successfully." #launch实例可以查到日志，但是attach volume无此日志 2021-12-05 07:03:51.217 12467 INFO cinder.volume.api [req-188f8bd3-4fa9-483e-8884-27053b4eae46 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Attach volume completed successfully. #此处为launch实例，/var/log/cinder/api.log对应的Initialize volume日志 [root@controller ~]# cat /var/log/cinder/api.log |grep -i req-71add0be-4246-4a76-8c2c-dbc2f72ec7c4 2021-12-05 07:03:48.764 12467 INFO cinder.api.openstack.wsgi [req-71add0be-4246-4a76-8c2c-dbc2f72ec7c4 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] POST http://192.168.1.21:8776/v3/f911df6b22d54d34b716efd7a97edf92/volumes/cb3f11fc-162c-4846-acaa-1c37b1ef542f/action 2021-12-05 07:03:48.764 12467 DEBUG cinder.api.openstack.wsgi [req-71add0be-4246-4a76-8c2c-dbc2f72ec7c4 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Action body: {"os-initialize_connection": {"connector": {"platform": "x86_64", "host": "compute22", "system uuid": "B2E84D56-40A1-3B4E-11F3-62D43C5756EC", "do_local_attach": false, "ip": "192.168.1.22", "os_type": "linux2", "multipath": false, "initiator": "iqn.1994-05.com.redhat:51f266169dc3"}}} get_method /usr/lib/python2.7/site-packages/cinder/api/openstack/wsgi.py:993 2021-12-05 07:03:48.765 12467 DEBUG cinder.api.openstack.wsgi [req-71add0be-4246-4a76-8c2c-dbc2f72ec7c4 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Action: 'action', calling method: _initialize_connection, body: {"os-initialize_connection": {"connector": {"platform": "x86_64", "host": "compute22", "system uuid": "B2E84D56-40A1-3B4E-11F3-62D43C5756EC", "do_local_attach": false, "ip": "192.168.1.22", "os_type": "linux2", "multipath": false, "initiator": "iqn.1994-05.com.redhat:51f266169dc3"}}} _process_stack /usr/lib/python2.7/site-packages/cinder/api/openstack/wsgi.py:876 2021-12-05 07:03:48.791 12467 INFO cinder.volume.api [req-71add0be-4246-4a76-8c2c-dbc2f72ec7c4 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Volume info retrieved successfully. 2021-12-05 07:03:51.030 12467 INFO cinder.volume.api [req-71add0be-4246-4a76-8c2c-dbc2f72ec7c4 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Initialize volume connection completed successfully. 2021-12-05 07:03:51.030 12467 INFO cinder.api.openstack.wsgi [req-71add0be-4246-4a76-8c2c-dbc2f72ec7c4 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] http://192.168.1.21:8776/v3/f911df6b22d54d34b716efd7a97edf92/volumes/cb3f11fc-162c-4846-acaa-1c37b1ef542f/action returned with HTTP 200 2021-12-05 07:03:51.031 12467 INFO eventlet.wsgi.server [req-71add0be-4246-4a76-8c2c-dbc2f72ec7c4 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] 192.168.1.22 "POST /v3/f911df6b22d54d34b716efd7a97edf92/volumes/cb3f11fc-162c-4846-acaa-1c37b1ef542f/action HTTP/1.1" status: 200 len: 773 time: 2.2692750 #此处为launch实例，/var/log/cinder/api.log对应的Attach volume日志 [root@controller ~]# cat /var/log/cinder/api.log |grep -i "req-188f8bd3-4fa9-483e-8884-27053b4eae46" 2021-12-05 07:03:51.052 12467 INFO cinder.api.openstack.wsgi [req-188f8bd3-4fa9-483e-8884-27053b4eae46 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] POST http://192.168.1.21:8776/v3/f911df6b22d54d34b716efd7a97edf92/volumes/cb3f11fc-162c-4846-acaa-1c37b1ef542f/action 2021-12-05 07:03:51.053 12467 DEBUG cinder.api.openstack.wsgi [req-188f8bd3-4fa9-483e-8884-27053b4eae46 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Action body: {"os-attach": {"instance_uuid": "fd513532-4a76-480f-a64d-a1e842901296", "mountpoint": "/dev/vda", "mode": "rw"}} get_method /usr/lib/python2.7/site-packages/cinder/api/openstack/wsgi.py:993 2021-12-05 07:03:51.053 12467 DEBUG cinder.api.openstack.wsgi [req-188f8bd3-4fa9-483e-8884-27053b4eae46 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Action: 'action', calling method: _attach, body: {"os-attach": {"instance_uuid": "fd513532-4a76-480f-a64d-a1e842901296", "mountpoint": "/dev/vda", "mode": "rw"}} _process_stack /usr/lib/python2.7/site-packages/cinder/api/openstack/wsgi.py:876 2021-12-05 07:03:51.080 12467 INFO cinder.volume.api [req-188f8bd3-4fa9-483e-8884-27053b4eae46 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Volume info retrieved successfully. 2021-12-05 07:03:51.091 12467 INFO cinder.volume.api [req-188f8bd3-4fa9-483e-8884-27053b4eae46 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Update volume admin metadata completed successfully. 2021-12-05 07:03:51.217 12467 INFO cinder.volume.api [req-188f8bd3-4fa9-483e-8884-27053b4eae46 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Attach volume completed successfully. 2021-12-05 07:03:51.218 12467 INFO cinder.api.openstack.wsgi [req-188f8bd3-4fa9-483e-8884-27053b4eae46 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] http://192.168.1.21:8776/v3/f911df6b22d54d34b716efd7a97edf92/volumes/cb3f11fc-162c-4846-acaa-1c37b1ef542f/action returned with HTTP 202 2021-12-05 07:03:51.218 12467 INFO eventlet.wsgi.server [req-188f8bd3-4fa9-483e-8884-27053b4eae46 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] 192.168.1.22 "POST /v3/f911df6b22d54d34b716efd7a97edf92/volumes/cb3f11fc-162c-4846-acaa-1c37b1ef542f/action HTTP/1.1" status: 202 len: 331 time: 0.1680911 2021-12-05 07:05:01.666 12467 INFO cinder.api.openstack.wsgi [req-188f8bd3-4fa9-483e-8884-27053b4eae46 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] GET http://192.168.1.21:8776/ 2021-12-05 07:05:01.666 12467 DEBUG cinder.api.openstack.wsgi [req-188f8bd3-4fa9-483e-8884-27053b4eae46 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Empty body provided in request get_body /usr/lib/python2.7/site-packages/cinder/api/openstack/wsgi.py:725 2021-12-05 07:05:01.666 12467 DEBUG cinder.api.openstack.wsgi [req-188f8bd3-4fa9-483e-8884-27053b4eae46 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Calling method 'all' _process_stack /usr/lib/python2.7/site-packages/cinder/api/openstack/wsgi.py:879 2021-12-05 07:05:01.667 12467 INFO cinder.api.openstack.wsgi [req-188f8bd3-4fa9-483e-8884-27053b4eae46 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] http://192.168.1.21:8776/ returned with HTTP 300 2021-12-05 07:05:01.667 12467 INFO eventlet.wsgi.server [req-188f8bd3-4fa9-483e-8884-27053b4eae46 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] 192.168.1.22 "GET / HTTP/1.1" status: 300 len: 963 time: 0.0019231 #此处为launch实例时cinder-volume初始化连接的log [root@controller ~]# cat /var/log/cinder/volume.log |grep req-71add0be-4246-4a76-8c2c-dbc2f72ec7c4 2021-12-05 07:03:48.798 12515 DEBUG oslo_concurrency.processutils [req-71add0be-4246-4a76-8c2c-dbc2f72ec7c4 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf lvchange -a y --yes -K cinder-volumes/volume-cb3f11fc-162c-4846-acaa-1c37b1ef542f execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:379 2021-12-05 07:03:49.075 12515 DEBUG oslo_concurrency.processutils [req-71add0be-4246-4a76-8c2c-dbc2f72ec7c4 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf lvchange -a y --yes -K cinder-volumes/volume-cb3f11fc-162c-4846-acaa-1c37b1ef542f" returned: 0 in 0.277s execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:416 2021-12-05 07:03:49.107 12515 INFO cinder.volume.targets.lio [req-71add0be-4246-4a76-8c2c-dbc2f72ec7c4 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Creating iscsi_target for volume: volume-cb3f11fc-162c-4846-acaa-1c37b1ef542f 2021-12-05 07:03:49.108 12515 DEBUG oslo_concurrency.lockutils [req-71add0be-4246-4a76-8c2c-dbc2f72ec7c4 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Lock "lioadm" acquired by "cinder.volume.targets.lio._execute" :: waited 0.001s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:327 2021-12-05 07:03:49.108 12515 DEBUG oslo_concurrency.processutils [req-71add0be-4246-4a76-8c2c-dbc2f72ec7c4 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf cinder-rtstool create /dev/cinder-volumes/volume-cb3f11fc-162c-4846-acaa-1c37b1ef542f iqn.2010-10.org.openstack:volume-cb3f11fc-162c-4846-acaa-1c37b1ef542f NUaraQHsGYNvJSdu949N fp9zdSY5N6yGCzMe False -p3260 -a192.168.1.21 execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:379 2021-12-05 07:03:49.491 12515 DEBUG oslo_concurrency.processutils [req-71add0be-4246-4a76-8c2c-dbc2f72ec7c4 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf cinder-rtstool create /dev/cinder-volumes/volume-cb3f11fc-162c-4846-acaa-1c37b1ef542f iqn.2010-10.org.openstack:volume-cb3f11fc-162c-4846-acaa-1c37b1ef542f NUaraQHsGYNvJSdu949N fp9zdSY5N6yGCzMe False -p3260 -a192.168.1.21" returned: 0 in 0.383s execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:416 2021-12-05 07:03:49.492 12515 DEBUG oslo_concurrency.lockutils [req-71add0be-4246-4a76-8c2c-dbc2f72ec7c4 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Lock "lioadm" released by "cinder.volume.targets.lio._execute" :: held 0.384s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:339 2021-12-05 07:03:49.492 12515 DEBUG oslo_concurrency.lockutils [req-71add0be-4246-4a76-8c2c-dbc2f72ec7c4 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Lock "lioadm" acquired by "cinder.volume.targets.lio._execute" :: waited 0.000s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:327 2021-12-05 07:03:49.493 12515 DEBUG oslo_concurrency.processutils [req-71add0be-4246-4a76-8c2c-dbc2f72ec7c4 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf cinder-rtstool get-targets execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:379 2021-12-05 07:03:49.854 12515 DEBUG oslo_concurrency.processutils [req-71add0be-4246-4a76-8c2c-dbc2f72ec7c4 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf cinder-rtstool get-targets" returned: 0 in 0.361s execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:416 2021-12-05 07:03:49.855 12515 DEBUG oslo_concurrency.lockutils [req-71add0be-4246-4a76-8c2c-dbc2f72ec7c4 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Lock "lioadm" released by "cinder.volume.targets.lio._execute" :: held 0.363s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:339 2021-12-05 07:03:49.856 12515 DEBUG oslo_concurrency.lockutils [req-71add0be-4246-4a76-8c2c-dbc2f72ec7c4 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Lock "lioadm" acquired by "cinder.volume.targets.lio._execute" :: waited 0.000s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:327 2021-12-05 07:03:49.856 12515 DEBUG oslo_concurrency.processutils [req-71add0be-4246-4a76-8c2c-dbc2f72ec7c4 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf cinder-rtstool save execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:379 2021-12-05 07:03:50.249 12515 DEBUG oslo_concurrency.processutils [req-71add0be-4246-4a76-8c2c-dbc2f72ec7c4 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf cinder-rtstool save" returned: 0 in 0.392s execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:416 2021-12-05 07:03:50.249 12515 DEBUG oslo_concurrency.lockutils [req-71add0be-4246-4a76-8c2c-dbc2f72ec7c4 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Lock "lioadm" released by "cinder.volume.targets.lio._execute" :: held 0.394s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:339 2021-12-05 07:03:50.250 12515 DEBUG cinder.volume.targets.iscsi [req-71add0be-4246-4a76-8c2c-dbc2f72ec7c4 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Set provider_location to: 192.168.1.21:3260,iqn.2010-10.org.openstack:volume-cb3f11fc-162c-4846-acaa-1c37b1ef542f iqn.2010-10.org.openstack:volume-cb3f11fc-162c-4846-acaa-1c37b1ef542f 0 create_export /usr/lib/python2.7/site-packages/cinder/volume/targets/iscsi.py:213 2021-12-05 07:03:50.257 12515 DEBUG oslo_concurrency.lockutils [req-71add0be-4246-4a76-8c2c-dbc2f72ec7c4 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Lock "lioadm" acquired by "cinder.volume.targets.lio._execute" :: waited 0.000s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:327 2021-12-05 07:03:50.258 12515 DEBUG oslo_concurrency.processutils [req-71add0be-4246-4a76-8c2c-dbc2f72ec7c4 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf cinder-rtstool add-initiator iqn.2010-10.org.openstack:volume-cb3f11fc-162c-4846-acaa-1c37b1ef542f NUaraQHsGYNvJSdu949N fp9zdSY5N6yGCzMe iqn.1994-05.com.redhat:51f266169dc3 execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:379 2021-12-05 07:03:50.619 12515 DEBUG oslo_concurrency.processutils [req-71add0be-4246-4a76-8c2c-dbc2f72ec7c4 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf cinder-rtstool add-initiator iqn.2010-10.org.openstack:volume-cb3f11fc-162c-4846-acaa-1c37b1ef542f NUaraQHsGYNvJSdu949N fp9zdSY5N6yGCzMe iqn.1994-05.com.redhat:51f266169dc3" returned: 0 in 0.361s execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:416 2021-12-05 07:03:50.620 12515 DEBUG oslo_concurrency.lockutils [req-71add0be-4246-4a76-8c2c-dbc2f72ec7c4 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Lock "lioadm" released by "cinder.volume.targets.lio._execute" :: held 0.363s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:339 2021-12-05 07:03:50.621 12515 DEBUG oslo_concurrency.lockutils [req-71add0be-4246-4a76-8c2c-dbc2f72ec7c4 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Lock "lioadm" acquired by "cinder.volume.targets.lio._execute" :: waited 0.000s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:327 2021-12-05 07:03:50.621 12515 DEBUG oslo_concurrency.processutils [req-71add0be-4246-4a76-8c2c-dbc2f72ec7c4 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf cinder-rtstool save execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:379 2021-12-05 07:03:51.009 12515 DEBUG oslo_concurrency.processutils [req-71add0be-4246-4a76-8c2c-dbc2f72ec7c4 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf cinder-rtstool save" returned: 0 in 0.387s execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:416 2021-12-05 07:03:51.010 12515 DEBUG oslo_concurrency.lockutils [req-71add0be-4246-4a76-8c2c-dbc2f72ec7c4 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Lock "lioadm" released by "cinder.volume.targets.lio._execute" :: held 0.389s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:339 2021-12-05 07:03:51.026 12515 INFO cinder.volume.manager [req-71add0be-4246-4a76-8c2c-dbc2f72ec7c4 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Initialize volume connection completed successfully. #此处为launch实例时attach volume的log [root@controller ~]# cat /var/log/cinder/volume.log |grep req-188f8bd3-4fa9-483e-8884-27053b4eae46 2021-12-05 07:03:51.097 12515 DEBUG cinder.coordination [req-188f8bd3-4fa9-483e-8884-27053b4eae46 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Lock "/var/lib/cinder/cinder-cb3f11fc-162c-4846-acaa-1c37b1ef542f" acquired by "attach_volume" :: waited 0.000s _synchronized /usr/lib/python2.7/site-packages/cinder/coordination.py:150 2021-12-05 07:03:51.156 12515 INFO cinder.volume.manager [req-188f8bd3-4fa9-483e-8884-27053b4eae46 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Attaching volume cb3f11fc-162c-4846-acaa-1c37b1ef542f to instance fd513532-4a76-480f-a64d-a1e842901296 at mountpoint /dev/vda on host None. 2021-12-05 07:03:51.213 12515 INFO cinder.volume.manager [req-188f8bd3-4fa9-483e-8884-27053b4eae46 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Attach volume completed successfully. 2021-12-05 07:03:51.213 12515 DEBUG cinder.coordination [req-188f8bd3-4fa9-483e-8884-27053b4eae46 80f373269d7a481b931a9352b42b28b8 f911df6b22d54d34b716efd7a97edf92 - default default] Lock "/var/lib/cinder/cinder-cb3f11fc-162c-4846-acaa-1c37b1ef542f" released by "attach_volume" :: held 0.117s _synchronized /usr/lib/python2.7/site-packages/cinder/coordination.py:162
cinder剩余待更新
049 - 准备 LVM Volume Provider
055 - Detach Volume 操作
056 - Extend Volume 操作
057 - Delete Volume 操作
058 - Snapshot Volume 操作
059 - Backup Volume 操作
060 - Restore Volume 操作
061 - Boot from Volume
062 - NFS Volume Provider（Part I）
063 - NFS Volume Provider（Part II）
064 - NFS Volume Provider（Part III）
【推荐】国内首个AI IDE，深度理解中文开发场景，立即下载体验Trae
【推荐】编程新体验，更懂你的AI，立即体验豆包MarsCode编程助手
【推荐】抖音旗下AI助手豆包，你的智能百科全书，全免费不限次数
【推荐】轻量又高性能的 SSH 工具 IShell：AI 加持，快人一步
· SQL Server 2025 AI相关能力初探
· Linux系列：如何用 C#调用 C方法造成内存泄露
· AI与.NET技术实操系列（二）：开始使用ML.NET
· 记一次.NET内存居高不下排查解决与启示
· 探究高空视频全景AR技术的实现原理
· 阿里最新开源QwQ-32B，效果媲美deepseek-r1满血版，部署成本又又又降低了！
· 单线程的Redis速度为什么快？
· SQL Server 2025 AI相关能力初探
· AI编程工具终极对决：字节Trae VS Cursor，谁才是开发者新宠？
· 展开说说关于C#中ORM框架的用法！