openstack--5--控制节点和计算节点安装配置nova
Nova相关介绍
目前的Nova主要由API,Compute,Conductor,Scheduler组成
Compute:用来交互并管理虚拟机的生命周期;
Scheduler:从可用池中根据各种策略选择最合适的计算节点来创建新的虚拟机;
Conductor:为数据库的访问提供统一的接口层。
Compute Service Nova 是 OpenStack 最核心的服务,负责维护和管理云环境的计算资源。
OpenStack 作为 IaaS 的云操作系统,虚拟机生命周期管理也就是通过 Nova 来实现的
在上图中可以看到,Nova 处于 Openstak 架构的中心,其他组件都为 Nova 提供支持:
Glance 为 VM 提供 image
Cinder 和 Swift 分别为 VM 提供块存储和对象存储
Neutron 为 VM 提供网络连接
Nova 架构如下
Nova 的架构比较复杂,包含很多组件。
这些组件以子服务(后台 deamon 进程)的形式运行,可以分为以下几类:
AP部分
nova-api
其它模块的通过HTTP协议进入Nova模块的接口,api是外部访问nova的唯一途径
Compute Core部分
nova-scheduler
虚机调度服务,负责决定在哪个计算节点上运行虚机
Nova scheduler模块决策一个虚拟机应该调度到某个物理节点,需要两个步骤
1、过滤(Fliter):先过滤出符合条件的主机,空闲资源足够的。
2、计算权重
nova-compute
管理虚机的核心服务,通过调用 Hypervisor API 实现虚机生命周期管理
Hypervisor
计算节点上跑的虚拟化管理程序,虚机管理最底层的程序。 不同虚拟化技术提供自己的 Hypervisor。 常用的 Hypervisor 有 KVM,Xen, VMWare 等
nova-conductor
nova-compute 经常需要更新数据库,比如更新虚机的状态。 出于安全性和伸缩性的考虑,nova-compute 并不会直接访问数据库(早期版本计算节点可以直接访问数据库,计算节点一旦被入侵,整个库都危险)
而是将这个任务委托给 nova-conductor,这个我们在后面会详细讨论。
Console Interface部分
nova-console
用户可以通过多种方式访问虚机的控制台: nova-novncproxy,基于 Web 浏览器的 VNC 访问 nova-spicehtml5proxy,基于 HTML5 浏览器的 SPICE 访问 nova-xvpnvncproxy,基于 Java 客户端的 VNC 访问
nova-consoleauth
负责对访问虚机控制台请亲提供 Token 认证
nova-cert
Database部分
Nova 会有一些数据需要存放到数据库中,一般使用 MySQL。 Nova 使用名为 "nova" 和"nova_api"的数据库。
某个主机硬件资源足够并不表示符合Nova scheduler。
可能认为会没有资源的场景:
1、网络故障,主机网络节点有问题
2、nova compute故障了
控制节点安装和配置Nova
1、控制节点安装软件包
[root@linux-node1 ~]# yum install openstack-nova-api openstack-nova-conductor openstack-nova-console openstack-nova-novncproxy openstack-nova-scheduler -y Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.163.com * epel: mirror.premi.st * extras: mirrors.163.com * updates: mirrors.163.com Package 1:openstack-nova-api-13.1.2-1.el7.noarch already installed and latest version Package 1:openstack-nova-conductor-13.1.2-1.el7.noarch already installed and latest version Package 1:openstack-nova-console-13.1.2-1.el7.noarch already installed and latest version Package 1:openstack-nova-novncproxy-13.1.2-1.el7.noarch already installed and latest version Package 1:openstack-nova-scheduler-13.1.2-1.el7.noarch already installed and latest version Nothing to do [root@linux-node1 ~]#
2、配置部分---数据库连接配置
/etc/nova/nova.conf
在[api_database]和[database]部分,配置数据库的连接:
[api_database] ... connection = mysql+pymysql://nova:NOVA_DBPASS@controller/nova_api [database] ... connection = mysql+pymysql://nova:NOVA_DBPASS@controller/nova
更改结果如下
[root@linux-node1 ~]# grep -n '^[a-Z]' /etc/nova/nova.conf 2168:connection = mysql+pymysql://nova:nova@192.168.56.11/nova_api 3128:connection = mysql+pymysql://nova:nova@192.168.56.11/nova [root@linux-node1 ~]#
同步数据到mysql库,出现警告也没关系
[root@linux-node1 ~]# su -s /bin/sh -c "nova-manage api_db sync" nova [root@linux-node1 ~]# su -s /bin/sh -c "nova-manage db sync" nova [root@linux-node1 ~]#
[root@linux-node1 ~]# mysql -h192.168.56.11 -unova -pnova -e "use nova;show tables;" +--------------------------------------------+ | Tables_in_nova | +--------------------------------------------+ | agent_builds | | aggregate_hosts | | aggregate_metadata | | aggregates | | allocations | | block_device_mapping | | bw_usage_cache | | cells | | certificates | | compute_nodes | | console_pools | | consoles | | dns_domains | | fixed_ips | | floating_ips | | instance_actions | | instance_actions_events | | instance_extra | | instance_faults | | instance_group_member | | instance_group_policy | | instance_groups | | instance_id_mappings | | instance_info_caches | | instance_metadata | | instance_system_metadata | | instance_type_extra_specs | | instance_type_projects | | instance_types | | instances | | inventories | | key_pairs | | migrate_version | | migrations | | networks | | pci_devices | | project_user_quotas | | provider_fw_rules | | quota_classes | | quota_usages | | quotas | | reservations | | resource_provider_aggregates | | resource_providers | | s3_images | | security_group_default_rules | | security_group_instance_association | | security_group_rules | | security_groups | | services | | shadow_agent_builds | | shadow_aggregate_hosts | | shadow_aggregate_metadata | | shadow_aggregates | | shadow_block_device_mapping | | shadow_bw_usage_cache | | shadow_cells | | shadow_certificates | | shadow_compute_nodes | | shadow_console_pools | | shadow_consoles | | shadow_dns_domains | | shadow_fixed_ips | | shadow_floating_ips | | shadow_instance_actions | | shadow_instance_actions_events | | shadow_instance_extra | | shadow_instance_faults | | shadow_instance_group_member | | shadow_instance_group_policy | | shadow_instance_groups | | shadow_instance_id_mappings | | shadow_instance_info_caches | | shadow_instance_metadata | | shadow_instance_system_metadata | | shadow_instance_type_extra_specs | | shadow_instance_type_projects | | shadow_instance_types | | shadow_instances | | shadow_key_pairs | | shadow_migrate_version | | shadow_migrations | | shadow_networks | | shadow_pci_devices | | shadow_project_user_quotas | | shadow_provider_fw_rules | | shadow_quota_classes | | shadow_quota_usages | | shadow_quotas | | shadow_reservations | | shadow_s3_images | | shadow_security_group_default_rules | | shadow_security_group_instance_association | | shadow_security_group_rules | | shadow_security_groups | | shadow_services | | shadow_snapshot_id_mappings | | shadow_snapshots | | shadow_task_log | | shadow_virtual_interfaces | | shadow_volume_id_mappings | | shadow_volume_usage_cache | | snapshot_id_mappings | | snapshots | | tags | | task_log | | virtual_interfaces | | volume_id_mappings | | volume_usage_cache | +--------------------------------------------+ [root@linux-node1 ~]# mysql -h192.168.56.11 -unova -pnova -e "use nova_api;show tables;" +--------------------+ | Tables_in_nova_api | +--------------------+ | build_requests | | cell_mappings | | flavor_extra_specs | | flavor_projects | | flavors | | host_mappings | | instance_mappings | | migrate_version | | request_specs | +--------------------+ [root@linux-node1 ~]#
3、 配置部分---keystone配置
在[DEFAULT] 和 [keystone_authtoken] 部分,配置认证服务访问:
[DEFAULT] ... auth_strategy = keystone [keystone_authtoken] ... auth_uri = http://192.168.56.11:5000 auth_url = http://192.168.56.11:35357 memcached_servers = 192.168.56.11:11211 auth_type = password project_domain_name = default user_domain_name = default project_name = service username = nova password = nova
4、配置部分---RabbitMQ配置
修改rabbitmq的配置,nova它们各个组件之间要用到
在 [DEFAULT] 和 [oslo_messaging_rabbit]部分,配置 RabbitMQ 消息队列访问:
[DEFAULT] ... rpc_backend = rabbit [oslo_messaging_rabbit] ... rabbit_host=192.168.56.11 rabbit_userid = openstack rabbit_password = openstack
下面也可以把端口部分注释去掉
5、配置部分---nova自身功能模块的配置
在[DEFAULT]部分,只启用计算和元数据API:
[DEFAULT] ... enabled_apis = osapi_compute,metadata
在 [DEFAULT 部分,这里就不配置了,以删除线标识
[DEFAULT]
...
my_ip = 10.0.0.11
在 [DEFAULT] 部分,使能 Networking 服务:
默认情况下,计算服务使用内置的防火墙服务。由于网络服务包含了防火墙服务
你必须使用nova.virt.firewall.NoopFirewallDriver防火墙服务来禁用掉计算服务内置的防火墙服务
[DEFAULT] ... use_neutron = True firewall_driver = nova.virt.firewall.NoopFirewallDriver
在[vnc]部分,配置VNC代理使用控制节点的管理接口IP地址 (这里原来是$my_ip):
[vnc] ... vncserver_listen = 192.168.56.11 vncserver_proxyclient_address = 192.168.56.11
在 [glance] 区域,配置镜像服务 API 的位置:
[glance] ... api_servers = http://192.168.56.11:9292
在 [oslo_concurrency] 部分,配置锁路径:
[oslo_concurrency] ... lock_path = /var/lib/nova/tmp
[root@linux-node1 ~]# grep -n '^[a-Z]' /etc/nova/nova.conf 267:enabled_apis=osapi_compute,metadata 382:auth_strategy=keystone 1561:firewall_driver=nova.virt.libvirt.firewall.IptablesFirewallDriver 1684:use_neutron=True 2119:rpc_backend=rabbit 2168:connection = mysql+pymysql://nova:nova@192.168.56.11/nova_api 3128:connection = mysql+pymysql://nova:nova@192.168.56.11/nova 3354:api_servers= http://192.168.56.11:9292 3523:auth_uri = http://192.168.56.11:5000 3524:auth_url = http://192.168.56.11:35357 3525:memcached_servers = 192.168.56.11:11211 3526:auth_type = password 3527:project_domain_name = default 3528:user_domain_name = default 3529:project_name = service 3530:username = nova 3531:password = nova 4307:lock_path=/var/lib/nova/tmp 4458:rabbit_host=192.168.56.11 4464:rabbit_port=5672 4476:rabbit_userid=openstack 4480:rabbit_password=openstack 5427:vncserver_listen=192.168.56.11 5451:vncserver_proxyclient_address=192.168.56.11 [root@linux-node1 ~]#
7、启动nova服务
启动 nova 服务并将其设置为随系统启动
命令如下
systemctl enable openstack-nova-api.service \ openstack-nova-consoleauth.service openstack-nova-scheduler.service \ openstack-nova-conductor.service openstack-nova-novncproxy.service systemctl start openstack-nova-api.service \ openstack-nova-consoleauth.service openstack-nova-scheduler.service \ openstack-nova-conductor.service openstack-nova-novncproxy.service
执行过程如下
[root@linux-node1 ~]# systemctl enable openstack-nova-api.service \ > openstack-nova-consoleauth.service openstack-nova-scheduler.service \ > openstack-nova-conductor.service openstack-nova-novncproxy.service Created symlink from /etc/systemd/system/multi-user.target.wants/openstack-nova-api.service to /usr/lib/systemd/system/openstack-nova-api.service. Created symlink from /etc/systemd/system/multi-user.target.wants/openstack-nova-consoleauth.service to /usr/lib/systemd/system/openstack-nova-consoleauth.service. Created symlink from /etc/systemd/system/multi-user.target.wants/openstack-nova-scheduler.service to /usr/lib/systemd/system/openstack-nova-scheduler.service. Created symlink from /etc/systemd/system/multi-user.target.wants/openstack-nova-conductor.service to /usr/lib/systemd/system/openstack-nova-conductor.service. Created symlink from /etc/systemd/system/multi-user.target.wants/openstack-nova-novncproxy.service to /usr/lib/systemd/system/openstack-nova-novncproxy.service. [root@linux-node1 ~]# systemctl start openstack-nova-api.service \ > openstack-nova-consoleauth.service openstack-nova-scheduler.service \ > openstack-nova-conductor.service openstack-nova-novncproxy.service [root@linux-node1 ~]#
下面步骤之前已经操作了,这里就不做了,以删除线标识
1、获得 admin 凭证来获取只有管理员能执行的命令的访问权限
source admin-openrc
2、 要创建服务证书,完成这些步骤
创建 nova 用户
openstack user create --domain default \
--password-prompt nova
3、给 nova 用户添加 admin 角色
$ openstack role add --project service --user nova admin
openstack service create --name nova --description "OpenStack Compute" compute
执行过程如下
[root@linux-node1 ~]# source admin-openstack.sh [root@linux-node1 ~]# openstack service create --name nova --description "OpenStack Compute" compute +-------------+----------------------------------+ | Field | Value | +-------------+----------------------------------+ | description | OpenStack Compute | | enabled | True | | id | e1e90d1948fb4384a8d2b09edb2c0cf6 | | name | nova | | type | compute | +-------------+----------------------------------+ [root@linux-node1 ~]#
openstack endpoint create --region RegionOne compute public http://192.168.56.11:8774/v2.1/%\(tenant_id\)s
[root@linux-node1 ~]# openstack endpoint create --region RegionOne compute public http://192.168.56.11:8774/v2.1/%\(tenant_id\)s +--------------+----------------------------------------------+ | Field | Value | +--------------+----------------------------------------------+ | enabled | True | | id | 7017bf5b4990451296c6b51aff13e6f4 | | interface | public | | region | RegionOne | | region_id | RegionOne | | service_id | e1e90d1948fb4384a8d2b09edb2c0cf6 | | service_name | nova | | service_type | compute | | url | http://192.168.56.11:8774/v2.1/%(tenant_id)s | +--------------+----------------------------------------------+ [root@linux-node1 ~]#
创建internal端点
命令如下
openstack endpoint create --region RegionOne compute internal http://192.168.56.11:8774/v2.1/%\(tenant_id\)s
操作如下
[root@linux-node1 ~]# openstack endpoint create --region RegionOne compute internal http://192.168.56.11:8774/v2.1/%\(tenant_id\)s +--------------+----------------------------------------------+ | Field | Value | +--------------+----------------------------------------------+ | enabled | True | | id | b3325481dd704aeb94c02c48b97e0991 | | interface | internal | | region | RegionOne | | region_id | RegionOne | | service_id | e1e90d1948fb4384a8d2b09edb2c0cf6 | | service_name | nova | | service_type | compute | | url | http://192.168.56.11:8774/v2.1/%(tenant_id)s | +--------------+----------------------------------------------+ [root@linux-node1 ~]#
创建admin端点
命令如下
openstack endpoint create --region RegionOne compute admin http://192.168.56.11:8774/v2.1/%\(tenant_id\)s
操作如下
[root@linux-node1 ~]# openstack endpoint create --region RegionOne compute admin http://192.168.56.11:8774/v2.1/%\(tenant_id\)s +--------------+----------------------------------------------+ | Field | Value | +--------------+----------------------------------------------+ | enabled | True | | id | 8ce0f94f658141ce94aa339b43c48eea | | interface | admin | | region | RegionOne | | region_id | RegionOne | | service_id | e1e90d1948fb4384a8d2b09edb2c0cf6 | | service_name | nova | | service_type | compute | | url | http://192.168.56.11:8774/v2.1/%(tenant_id)s | +--------------+----------------------------------------------+ [root@linux-node1 ~]#
检查创建结果
[root@linux-node1 ~]# openstack host list +----------------------+-------------+----------+ | Host Name | Service | Zone | +----------------------+-------------+----------+ | linux-node1.nmap.com | scheduler | internal | | linux-node1.nmap.com | consoleauth | internal | | linux-node1.nmap.com | conductor | internal | +----------------------+-------------+----------+ [root@linux-node1 ~]#
计算节点安装和配置nova
nova compute通过libvirt管理kvm,计算节点是真正运行虚拟机的
计算节点机器必须打开vt-x
计算节点操作之前先同步下时间
[root@linux-node2 ~]# ntpdate time1.aliyun.com 18 Feb 12:53:43 ntpdate[3184]: adjust time server 115.28.122.198 offset -0.005554 sec [root@linux-node2 ~]# date Sat Feb 18 12:53:45 CST 2017 [root@linux-node2 ~]#
1、安装软件包
[root@linux-node2 ~]# yum install openstack-nova-compute -y Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.aliyun.com * epel: mirrors.tuna.tsinghua.edu.cn * extras: mirrors.163.com * updates: mirrors.aliyun.com Package 1:openstack-nova-compute-13.1.2-1.el7.noarch already installed and latest version Nothing to do [root@linux-node2 ~]#
关于novncproxy
novncproxy的端口是6080 ,登录控制节点查看下
[root@linux-node1 ~]# netstat -lntp | grep 6080 tcp 0 0 0.0.0.0:6080 0.0.0.0:* LISTEN 13967/python2 [root@linux-node1 ~]# [root@linux-node1 ~]# ps aux | grep 13967 nova 13967 0.0 1.6 379496 66716 ? Ss 11:24 0:06 /usr/bin/python2 /usr/bin/nova-novncproxy --web /usr/share/novnc/ root 16309 0.0 0.0 112644 968 pts/0 S+ 13:07 0:00 grep --colour=auto 13967 [root@linux-node1 ~]#
2、计算节点配置文件修改
计算节点的配置文件更改的地方和控制节点几乎一致,因此可以把控制节点的配置文件拷贝过来使用,还需要修改个别地方
1、计算节点没有配置连接数据库。(其实拷贝过来不删数据库的配置也能运行正常,但是不规范)
2、计算节点vnc多配置一行
[root@linux-node1 ~]# ls /etc/nova/ -l total 224 -rw-r----- 1 root nova 3673 Oct 10 21:20 api-paste.ini -rw-r----- 1 root nova 184346 Feb 18 11:20 nova.conf -rw-r----- 1 root nova 27914 Oct 10 21:20 policy.json -rw-r--r-- 1 root root 72 Oct 13 20:01 release -rw-r----- 1 root nova 966 Oct 10 21:20 rootwrap.conf [root@linux-node1 ~]#
[root@linux-node1 ~]# rsync -avz /etc/nova/nova.conf root@192.168.56.12:/etc/nova/ root@192.168.56.12's password: sending incremental file list sent 41 bytes received 12 bytes 15.14 bytes/sec total size is 184346 speedup is 3478.23 [root@linux-node1 ~]#
[root@linux-node2 ~]# ll /etc/nova/ total 224 -rw-r----- 1 root nova 3673 Oct 10 21:20 api-paste.ini -rw-r----- 1 root nova 184346 Feb 18 11:20 nova.conf -rw-r----- 1 root nova 27914 Oct 10 21:20 policy.json -rw-r--r-- 1 root root 72 Oct 13 20:01 release -rw-r----- 1 root nova 966 Oct 10 21:20 rootwrap.conf [root@linux-node2 ~]#
在[vnc]部分,启用并配置远程控制台访问:
[vnc] ... enabled = True vncserver_listen = 0.0.0.0 vncserver_proxyclient_address = $my_ip novncproxy_base_url = http://controller:6080/vnc_auto.html
服务器组件监听所有的 IP 地址,而代理组件仅仅监听计算节点管理网络接口的 IP 地址。
基本的 URL 只是你可以使用 web 浏览器访问位于该计算节点上实例的远程控制台的位置。
改成如下
[root@linux-node2 ~]# egrep -c '(vmx|svm)' /proc/cpuinfo 4 [root@linux-node2 ~]#
如果这个命令返回了 one or greater 的值,那么你的计算节点支持硬件加速且不需要额外的配置。
如果这个命令返回了 zero 值,那么你的计算节点不支持硬件加速。你必须配置 libvirt 来使用 QEMU 去代替 KVM
在 /etc/nova/nova.conf 文件的 [libvirt] 区域做出如下的编辑:
[libvirt] ... virt_type = qemu
外把下面注释去掉
[libvirt]模块部分也修改完毕
[root@linux-node2 ~]# grep -n '^[a-Z]' /etc/nova/nova.conf 267:enabled_apis=osapi_compute,metadata 382:auth_strategy=keystone 1561:firewall_driver=nova.virt.libvirt.firewall.IptablesFirewallDriver 1684:use_neutron=True 2119:rpc_backend=rabbit 3354:api_servers= http://192.168.56.11:9292 3523:auth_uri = http://192.168.56.11:5000 3524:auth_url = http://192.168.56.11:35357 3525:memcached_servers = 192.168.56.11:11211 3526:auth_type = password 3527:project_domain_name = default 3528:user_domain_name = default 3529:project_name = service 3530:username = nova 3531:password = nova 3683:virt_type=kvm 4307:lock_path=/var/lib/nova/tmp 4458:rabbit_host=192.168.56.11 4464:rabbit_port=5672 4476:rabbit_userid=openstack 4480:rabbit_password=openstack 5385:enabled=true 5427:vncserver_listen=0.0.0.0 5451:vncserver_proxyclient_address=192.168.56.12 5532:novncproxy_base_url=http://192.168.56.11:6080/vnc_auto.html [root@linux-node2 ~]#
3、启动服务和检查状态
启动计算服务及其依赖,并将其配置为随系统自动启动
[root@linux-node2 ~]# systemctl enable libvirtd.service openstack-nova-compute.service Created symlink from /etc/systemd/system/multi-user.target.wants/openstack-nova-compute.service to /usr/lib/systemd/system/openstack-nova-compute.service. [root@linux-node2 ~]# systemctl start libvirtd.service openstack-nova-compute.service [root@linux-node2 ~]#
[root@linux-node1 ~]# source admin-openstack.sh [root@linux-node1 ~]# openstack host list +----------------------+-------------+----------+ | Host Name | Service | Zone | +----------------------+-------------+----------+ | linux-node1.nmap.com | scheduler | internal | | linux-node1.nmap.com | consoleauth | internal | | linux-node1.nmap.com | conductor | internal | | linux-node2.nmap.com | compute | nova | +----------------------+-------------+----------+ [root@linux-node1 ~]#
控制节点列出nova的服务,后面的update时间都几乎一致,如果差距过大,可能造成无法创建虚拟机
[root@linux-node1 ~]# nova service-list +----+------------------+----------------------+----------+---------+-------+----------------------------+-----------------+ | Id | Binary | Host | Zone | Status | State | Updated_at | Disabled Reason | +----+------------------+----------------------+----------+---------+-------+----------------------------+-----------------+ | 1 | nova-scheduler | linux-node1.nmap.com | internal | enabled | up | 2017-02-18T05:41:14.000000 | - | | 2 | nova-consoleauth | linux-node1.nmap.com | internal | enabled | up | 2017-02-18T05:41:14.000000 | - | | 3 | nova-conductor | linux-node1.nmap.com | internal | enabled | up | 2017-02-18T05:41:13.000000 | - | | 7 | nova-compute | linux-node2.nmap.com | nova | enabled | up | 2017-02-18T05:41:13.000000 | - | +----+------------------+----------------------+----------+---------+-------+----------------------------+-----------------+ [root@linux-node1 ~]#
下面命令测试nova连接glance是否正常
[root@linux-node1 ~]# nova image-list +--------------------------------------+--------+--------+--------+ | ID | Name | Status | Server | +--------------------------------------+--------+--------+--------+ | 9969eaa3-0296-48cc-a42e-a02251b778a6 | cirros | ACTIVE | | +--------------------------------------+--------+--------+--------+ [root@linux-node1 ~]#