RHCA cl210 015 实例启动 超融合 热迁移 网络underlay
lab computeresources-hci setup
实例启动流程
keystone不仅做认证，且有所有组键地址
nova-conductor解耦，不允许nova-compute直接访问database
学习方法
学会画逻辑图，且看图比看文字来的要快。文字可能会帮助理解
将学会的东西说出来
超融合架构
存储+计算
nova-compute发放云主机 cpu 内存 网络全使用当前计算节点的资源
实例的磁盘来自单独的ceph集群
超融合既可以跑实例又可以存磁盘数据
红帽提供两种超融合架构
RHV+RHGS
RHOSP13+RHCS3
超融合为分布式 一部分属于ceph 一部分属于openstack
部署角色
(undercloud) [stack@director templates]$ pwd
/home/stack/templates
(undercloud) [stack@director templates]$ ls roles_data.yaml
环境变量 34.定义各个网段的ip
(undercloud) [stack@director classroom-environment]$ ls
00-node-info.yaml 32-network-environment.yaml
02-custom-domain.yaml 34-ips-from-pool-all.yaml
04-custom-hostnames.yaml 36-fixed-ip-vips.yaml
10-inject-trust-anchor.yaml 40-octavia.yaml
12-low-memory-usage.yaml 42-neutron-ovn-dvr-ha.yaml
14-disable-telemetry.yaml 50-ceph-ansible.yaml
16-telemetry-optimization.yaml 52-ceph-config.yaml
18-compute-extraconfig.yaml 54-ceph-mds.yaml
22-change_root_password-env.yaml 55-manila-cephfsnative-config.yaml
24-admin-password.yaml 90-local_registry_images.yaml
30-network-isolation.yaml 95-idm-integration.yaml
(undercloud) [stack@director classroom-environment]$ pwd
/home/stack/templates/classroom-environment
(undercloud) [stack@director classroom-environment]$
超融合将hypervisor计算角色与同一节点上的本地Ceph OSD相整合
超融合
计算资源和存储资源会互相争夺资源
(计算服务感受不到ceph也在使用资源)
所以需要设置资源约束
工作负载参数
reserved_host_memory 为计算资源预留多少资源
例如 一个节点具有256G内存和10个OSD，每个OSD通常消耗3GB内存，其中分配30GB用于ceph，其余226GB用于计算，这就能托管113个分别使用2GB内存的实例
cpu_allocation_ration参数
默认值16
设置低一点，消耗就会低一点
如果你cpu上限高，这个值就可以设置高一点。更好的压榨cpu
ceph回填
这是需要资源的，osd损坏了的话，会在其他osd恢复，osd会重平衡
超融合实验
用管理员用户添加主机集合
切换普通用户创建实例
选no不需要他创建1个g的卷
创建实例后，直接报错？
not valid host was found
回到管理员
删除主机集合
超融合需要配合flavor
创建flavor并设置metadata
主句聚合有
AZ与Host Aggregates
不输入AZ就不会创建新的AZ
添加超融合节点后更新主机组元数据
完成更新主机聚合的元数据
管理员才能查看hci在哪个主机上
[student@workstation ~(architect1-finance)]$ openstack server list
+--------------------------------------+------+--------+----------+-------+--------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+------+--------+----------+-------+--------+
| 8cab1c9b-7f03-41b1-81fc-f7980743cc8e | hci | BUILD | | | hci |
+--------------------------------------+------+--------+----------+-------+--------+
[student@workstation ~(architect1-finance)]$ openstack server show hci
+-------------------------------------+------------------------------------------------------------------+
| Field | Value |
+-------------------------------------+------------------------------------------------------------------+
| OS-DCF:diskConfig | AUTO |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-SRV-ATTR:host | compute1.overcloud.example.com |
| OS-EXT-SRV-ATTR:hypervisor_hostname | compute1.overcloud.example.com
有问题没有达到效果，云主机没有去到超融合节点
命令行再试试
管理员创建主机聚合
[student@workstation ~(architect1-finance)]$ openstack aggregate create hci-aggregate
+-------------------+----------------------------+
| Field | Value |
+-------------------+----------------------------+
| availability_zone | None |
| created_at | 2020-05-31T19:11:50.772175 |
| deleted | False |
| deleted_at | None |
| id | 5 |
| name | hci-aggregate |
| updated_at | None |
+-------------------+----------------------------+
[student@workstation ~(architect1-finance)]$ openstack aggregate add host hci-aggregate computehci0.overcloud.example.com
+-------------------+----------------------------------------+
| Field | Value |
+-------------------+----------------------------------------+
| availability_zone | None |
| created_at | 2020-05-31T19:11:50.000000 |
| deleted | False |
| deleted_at | None |
| hosts | [u'computehci0.overcloud.example.com'] |
| id | 5 |
| metadata | {} |
| name | hci-aggregate |
| updated_at | None |
+-------------------+----------------------------------------+
[student@workstation ~(architect1-finance)]$
添加元数据
[student@workstation ~(architect1-finance)]$ openstack aggregate set --property hci=true hci-aggregate
[student@workstation ~(architect1-finance)]$ openstack aggregate show hci-aggregate
+-------------------+----------------------------------------+
| Field | Value |
+-------------------+----------------------------------------+
| availability_zone | None |
| created_at | 2020-05-31T19:11:50.000000 |
| deleted | False |
| deleted_at | None |
| hosts | [u'computehci0.overcloud.example.com'] |
| id | 5 |
| name | hci-aggregate |
| properties | hci='true' |
| updated_at | None |
+-------------------+----------------------------------------+
创建flavor
[student@workstation ~(architect1-finance)]$ openstack flavor create --ram 1024 --disk 10 --vcpus 2 --public default-hci
+----------------------------+--------------------------------------+
| Field | Value |
+----------------------------+--------------------------------------+
| OS-FLV-DISABLED:disabled | False |
| OS-FLV-EXT-DATA:ephemeral | 0 |
| disk | 10 |
| id | d4b5695e-eca8-4c3e-bd1d-5e8e274ac6db |
| name | default-hci |
| os-flavor-access:is_public | True |
| properties | |
| ram | 1024 |
| rxtx_factor | 1.0 |
| swap | |
| vcpus | 2 |
+----------------------------+--------------------------------------+
添加元数据
[student@workstation ~(architect1-finance)]$ openstack flavor set --property hci=true default-hci
创建云主机 （也可也用普通用户developer创建）
[student@workstation ~(architect1-finance)]$ openstack server create --flavor default-hci --image hci-image --nic net-id=finance-network1 hci-server1
flavor得是1G内存才能发到超融合里。超融合没能力跑2G内存
主机聚合配合flavor，主机聚合与flavor都得打上一样的标签。flavor创建的虚拟机才会选择相关主机聚合中的主机
lab computeresources-hci setup
清环境
管理计算节点
准备环境
[student@workstation ~(architect1-finance)]$ lab computeresources-migrate setup
创建三个实例
冷迁移需要停业务，底层没有共享文件
热迁移
不中断业务迁移，不会察觉中断
两种方法，一种是块（需要时间迁移块），一种是共享存储vms（只需要迁移内存中的数据）
[heat-admin@controller0 ~]$ rbd -p vms ls
0111edda-900f-420e-8aae-0aaa44e44fae_disk
da04be11-1dbf-44a8-8e63-ca7d819157e5_disk
差分盘在同一个存储
热迁移，底层是要有母盘的，差分盘在共享文件里面，迁的是内存
有一个短暂的时间不接受请求。把那一点内存赶紧迁完
绑定外网ip
[student@workstation ~(architect1-finance)]$ openstack server add floating ip finance-server2 172.25.250.101
迁移完成且业务没有中断
命令行
openstack server migrate 主机 --live compute1.overcloud.example.com
[student@workstation ~(architect1-finance)]$ openstack server list --host compute1.overcloud.example.com --all-projects
+--------------------------------------+-----------------+--------+----------------------------------------------+-------+--------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+-----------------+--------+----------------------------------------------+-------+--------+
| 0111edda-900f-420e-8aae-0aaa44e44fae | finance-server2 | ACTIVE | finance-network1=192.168.1.8, 172.25.250.101 | rhel7 | m1.web |
+--------------------------------------+-----------------+--------+-----------------------------
当节点坏了，可以撤离。撤离后别人不能把业务调度到当前节点上
[student@workstation ~(architect1-finance)]$ openstack compute service list
+----+------------------+-----------------------------------+----------+---------+-------+----------------------------+
| ID | Binary | Host | Zone | Status | State | Updated At |
+----+------------------+-----------------------------------+----------+---------+-------+----------------------------+
| 1 | nova-conductor | controller0.overcloud.example.com | internal | enabled | up | 2020-05-31T17:53:30.000000 |
| 2 | nova-compute | compute1.overcloud.example.com | nova | enabled | up | 2020-05-31T17:53:27.000000 |
| 3 | nova-compute | computehci0.overcloud.example.com | nova | enabled | up | 2020-05-31T17:53:33.000000 |
| 4 | nova-compute | compute0.overcloud.example.com | nova | enabled | up | 2020-05-31T17:53:27.000000 |
| 5 | nova-consoleauth | controller0.overcloud.example.com | internal | enabled | up | 2020-05-31T17:53:29.000000 |
| 6 | nova-scheduler | controller0.overcloud.example.com | internal | enabled | up | 2020-05-31T17:53:33.000000 |
+----+------------------+-----------------------------------+----------+---------+-------+----------------------------+
得将节点改为当前不可用，禁用掉
[student@workstation ~(architect1-finance)]$ openstack compute service set compute1.overcloud.example.com nova-compute --disable
[student@workstation ~(architect1-finance)]$ openstack compute service list
+----+------------------+-----------------------------------+----------+----------+-------+----------------------------+
| ID | Binary | Host | Zone | Status | State | Updated At |
+----+------------------+-----------------------------------+----------+----------+-------+----------------------------+
| 1 | nova-conductor | controller0.overcloud.example.com | internal | enabled | up | 2020-05-31T17:55:20.000000 |
| 2 | nova-compute | compute1.overcloud.example.com | nova | disabled | up | 2020-05-31T17:55:20.000000 |
| 3 | nova-compute | computehci0.overcloud.example.com | nova | enabled | up | 2020-05-31T17:55:23.000000 |
| 4 | nova-compute | compute0.overcloud.example.com | nova | enabled | up | 2020-05-31T17:55:17.000000 |
| 5 | nova-consoleauth | controller0.overcloud.example.com | internal | enabled | up | 2020-05-31T17:55:19.000000 |
| 6 | nova-scheduler | controller0.overcloud.example.com | internal | enabled | up | 2020-05-31T17:55:23.000000 |
+----+------------------+-----------------------------------+----------+----------+-------+----------------------------+
驱逐节点上所有主机
[student@workstation ~(architect1-finance)]$ nova host-evacuate-live compute1.overcloud.example.com
+--------------------------------------+-------------------------+---------------+
| Server UUID | Live Migration Accepted | Error Message |
+--------------------------------------+-------------------------+---------------+
| 0111edda-900f-420e-8aae-0aaa44e44fae | True | |
+--------------------------------------+-------------------------+---------------+
[student@workstation ~(architect1-finance)]$ openstack compute service set compute1.overcloud.example.com nova-compute --enable
neutron
osp网络组键更新较大
SDN实现二层网络虚拟化 （以前ovs，ovs虚拟交换机） NFV实现三层及以上虚拟化
OVN 开放虚拟网络 属于openstack项目
OVN是RHOSP的默认SDN,SDN默认使用OVN
VXLAN Geneve
vlan做隔离 12位vlanid不够用
虚拟机迁移，可能要跨三层。vxlan为大二层，可以使云主机ip地址保持不变
vxlan的 vni隔离可能也不够有
geneve vxlan 都是用vni
geneve 除了增加vni数量，也有更好的适用能力
geneve 元数据可扩展
ovn可以实现防火墙与路由，ovn为ovs的升级 ovn使用geneve协议
vlan20 实例之间的传输
[root@controller0 ~]# vi /var/lib/config-data/neutron/etc/neutron/plugins/ml2/ml2_conf.ini
配置文件
[root@controller0 ~]# lsmod | grep nf_conntrack
nf_conntrack_netlink 36354 0
nfnetlink 14490 2 nf_conntrack_netlink
nf_conntrack_ipv6 18935 74
nf_defrag_ipv6 35104 2 openvswitch,nf_conntrack_ipv6
nf_conntrack_ipv4 15053 75
nf_defrag_ipv4 12729 1 nf_conntrack_ipv4
nf_conntrack 133053 8 openvswitch,nf_nat,nf_nat_ipv4,nf_nat_ipv6,xt_conntrack,nf_conntrack_netlink,nf_conntrack_ipv4,nf_conntrack_ipv6
libcrc32c 12644 4 xfs,openvswitch,nf_nat,nf_conntrack
[root@controller0 ~]# iptables
ovn桥可以直接调用 模块nf_conntrack
iptables 也用的nf_conntrack
每个节点都有一个分布式路由br-ex osp6只有一个br-ex并放在了网络节点
这样使同一个节点不同网段的实例可以互相访问
也可以使每个节点通过自己的路由去上网，不用去找网络节点
br-int可以直接调用nf_conntrack实现规则，br-int与br-int之间可以直接建立隧道
br-trunk
controller建立了三条隧道
ovs-vsctl-show
Bridge br-int
fail_mode: secure
Port "o-hm0"
Interface "o-hm0"
type: internal
Port "ovn-8b62a3-0"
Interface "ovn-8b62a3-0"
type: geneve
options: {csum="true", key=flow, remote_ip="172.24.2.2"}
Port "patch-br-int-to-provnet-fc5472ee-98d9-4f6b-9bc9-544ca18aefb3"
Interface "patch-br-int-to-provnet-fc5472ee-98d9-4f6b-9bc9-544ca18aefb3"
type: patch
options: {peer="patch-provnet-fc5472ee-98d9-4f6b-9bc9-544ca18aefb3-to-br-int"}
Port br-int
Interface br-int
type: internal
Port "ovn-028ad0-0"
Interface "ovn-028ad0-0"
type: geneve
options: {csum="true", key=flow, remote_ip="172.24.2.6"}
Port "ovn-cc9fe3-0"
Interface "ovn-cc9fe3-0"
type: geneve
options: {csum="true", key=flow, remote_ip="172.24.2.12"}
做内网通信的桥，以及安全组桥被合成了一个桥，br-int
各个不同平面的网络，应该在不同平面的网卡上配置ip
eth1被做成br-trunk桥 br-trunk为虚拟交换机vs。eth1为桥的端口
eth1又虚拟出多个子接口，每个子接口为一个ip 172.24.x.x
子接口通过br-trunk出去
eth1既被做成桥也被做成很多个子接口
这是其中一个子接口
[root@controller0 network-scripts]# cat ifcfg-vlan10
# This file is autogenerated by os-net-config
DEVICE=vlan10
ONBOOT=yes
HOTPLUG=no
NM_CONTROLLED=no
PEERDNS=no
DEVICETYPE=ovs
TYPE=OVSIntPort
OVS_BRIDGE=br-trunk
OVS_OPTIONS="tag=10"
BOOTPROTO=static
IPADDR=172.24.1.1
NETMASK=255.255.255.0
子接口通过trunk桥出去
不同网段连在同一个二层设备上。有网络风暴的风险，打上tag隔离广播
vlan20为租户网络
所以实例以后通过vlan20然后trunk出去通信
vlan20和vlan20之间打了隧道
所以第一层壳是172.24这个租户网络 打了tag不会广播，不打会广播 underlay网络
解封装后192，还有vni标签所以不会广播到其他实例
192.168.100.1 192.168.100.2两个不同主机通过vlan20这条隧道ping通
br-trunk可以没有，拿物理交互机可以替代
br-trunk underlay 通过这个出去
租户网络 overlay 解包后得到租户网络
S172.24.2.2
D172.24.2.1
tag隔离风暴
数据包解封装前
S192.168.100.1
D192.168.100.2
VNI隔离
解封装后
实例通过租户网络发数据包，数据包经过隧道（br-int）。重新封装为172租户地址，然后走vlan20，vlan20从trunk出去
