Floating IP in OpenStack Neutron
- 前言
Floating IP 是相对于Fixed IP而言的，它一般是在VM创建后分配给VM的，可以达到的目的就是，外界可以访问通过这个Floating Ip访问这个VM，VM也可以通过这个IP访问外界。
在OpenStack中，这个Floating IP使用了namespace内的iptables建立NAT 转发机制来达到VM与外界的通讯的。这片文章主要讲述如何使用OpenStack搭建和使用Floating IP.
- Environment Setup
- Ubuntu 14.04 LTS
- 2个网卡，分别是eth0（192.168.1.46） 和 eth1（192.168.2.46，这个interface在switch使用vlanid=100）
- 如果你的机器是VM，请参照http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1004099，去开启eth0 和eth1的promisic模式， 这一步很关键，如果没有启用，外界是无法ping通分配给VM的Floating IP的
- Devstack Configuration
- 启用Nova, Neutron, Cinder, Keystone, Glance
- 安装OpenvSwitch，然后做如下配置
-
# Add eth1 into br-eth1 and set eth0 and eth1 to promisc mode sudo ovs-vsctl br-exists br-eth1 || sudo ovs-vsctl add-br br-eth1 sudo ovs-vsctl --may-exist add-port br-eth1 eth1 sudo ip link set dev eth1 promisc on sudo ip link set dev eth0 promisc on sudo ip addr flush eth1 sudo ip link set dev eth1 up
- Devstack local.conf
-
# Note: please change XX to your devstack node IP, change VLAN_START and VLAN_END to your reserved vlan range Here # 我的IP是192.168.1.46， 所在网络是192.168.1.0/24 [[local|localrc]] # Set API endpoint host using HOST_IP HOST_IP=XX # Use to specify the endpoint SERVICE_HOST=XX ADMIN_PASSWORD=welcome MYSQL_PASSWORD=welcome RABBIT_PASSWORD=welcome SERVICE_PASSWORD=welcome SERVICE_TOKEN=welcome disable_service h-eng disable_service h-api disable_service h-api-cfn disable_service h-api-cw disable_service tempest #enable_service tempest disable_service dstat # Enable Neturon disable_service n-net enable_service q-svc enable_service q-agt enable_service q-dhcp enable_service q-l3 enable_service q-meta enable_service neutron # stack.sh will freshen each repo on each run if RECLONE # is set to yes RECLONE=True # Setting OFFLINE=True to enable stack.sh to run multiple # times without an Internet connection OFFLINE=False # Set FLAT_INTERFACE to the Ethernet interface that connects # the host to your local network FLAT_INTERFACE=eth1 #FLOATING_RANGE=192.168.1.0/24 #Q_FLOATING_ALLOCATION_POOL=start=192.168.1.100,end=192.168.1.119 #PUBLIC_NETWORK_GATEWAY=192.168.1.30 PUBLIC_INTERFACE=eth0 PHYSICAL_NETWORK=public_eth1 OVS_PHYSICAL_BRIDGE=br-ex #PUBLIC_BRIDGE=br-ex FIXED_RANGE=192.168.100.0/24 #FIXED_NETWORK_SIZE=32 NETWORK_GATEWAY=192.168.100.1 # IMAGE_URLS accepts a comma separated list of images to pre-load into OpenStack IMAGE_URLS=http://download.cirros-cloud.net/0.3.4/cirros-0.3.4-x86_64-disk.img [[post-config|$NOVA_CONF]] [DEFAULT] rpc_response_timeout=300 service_down_time=300 libvirt_iscsi_use_multipath = True #force_config_drive = False [libvirt] iscsi_use_multipath = True [database] max_pool_size=40 max_overflow=60 [[post-config|/$Q_PLUGIN_CONF_FILE]] [ml2] tenant_network_types = vlan,flat [ml2_type_flat] flat_networks = public_eth0 [ml2_type_vlan] network_vlan_ranges = public_eth1:VLAN_START:VLAN_END [ovs] bridge_mappings = public_eth0:br-ex,public_eth1:br-eth1 enable_tunneling = False
- Setup flat and vlan network
- 接下来，我要做两件事，一个是创建一个VLAN ID为100的private network，以后的虚拟机默认是从这个网络分配IP的。
- 然后是一个flat的public network，这个网络上是用来分配floating IP
创建private network，VLAN ID为100
stack@openstack-wangp11-01:~/devstack$ neutron net-create --provider:network_type vlan --provider:physical_network public_eth1 --provider:segmentation_id 100 vlan_100 stack@openstack-wangp11-01:~/devstack$ neutron net-show vlan_100 +---------------------------+--------------------------------------+ | Field | Value | +---------------------------+--------------------------------------+ | admin_state_up | True | | id | 2426ff0d-953f-467f-a564-c4f63d926836 | | mtu | 0 | | name | vlan_100 | | port_security_enabled | True | | provider:network_type | vlan | | provider:physical_network | public_eth1 | | provider:segmentation_id | 100 | | router:external | False | | shared | False | | status | ACTIVE | | subnets | fab94e2f-f7de-4bf6-8e9b-7ade0fcfad1f | | tenant_id | 8cb8c084ffb84914b41d5044ecbcad4e | +---------------------------+--------------------------------------+ stack@openstack-wangp11-01:~/devstack$ neutron subnet-create --enable-dhcp --ip-version 4 --name vlan_100_subnet01 2426ff0d-953f-467f-a564-c4f63d926836 192.168.46.0/24 stack@openstack-wangp11-01:~/devstack$ neutron subnet-show fab94e2f-f7de-4bf6-8e9b-7ade0fcfad1f +-------------------+----------------------------------------------------+ | Field | Value | +-------------------+----------------------------------------------------+ | allocation_pools | {"start": "192.168.46.2", "end": "192.168.46.254"} | | cidr | 192.168.46.0/24 | | dns_nameservers | | | enable_dhcp | True | | gateway_ip | 192.168.46.1 | | host_routes | | | id | fab94e2f-f7de-4bf6-8e9b-7ade0fcfad1f | | ip_version | 4 | | ipv6_address_mode | | | ipv6_ra_mode | | | name | vlan_100_subnet01 | | network_id | 2426ff0d-953f-467f-a564-c4f63d926836 | | subnetpool_id | | | tenant_id | 8cb8c084ffb84914b41d5044ecbcad4e | +-------------------+----------------------------------------------------+
接下来，创建Floating IP所在的public network, 网络类型是flat
注意下面：192.168.1.100-192.168.1.119是网络管理员分配给我的IP pool，192.168.1.30是我们网络的router ip
# Step 1: create external flat network $ neutron net-create --router:external --provider:network_type flat --provider:physical_network public_eth0 public_net # Step 2: create subnet based on your reservation Here $ neutron subnet-create --name public_eth0_subnet01 --allocation-pool start=192.168.1.100,end=192.168.1.119 --gateway 192.168.1.30 --disable-dhcp --ip-version 4 22d01ce7-b4c4-4af2-bc3c-9c3991903b4c 192.168.1.0/24 # Step 3: create a external router $ neutron router-create router1 # Step 4: connect router1 with the private subnet via 'neutron router-interface-add <router-id> <private_subnet_id>' $ neutron router-interface-add 2da62299-6e41-4b24-a1c7-a3d6cc4db1c8 fab94e2f-f7de-4bf6-8e9b-7ade0fcfad1f # Step 5: Set the external public network as the router gateway 'neutron router-gateway-set <router-id> <public_subnet_id>' $ neutron router-gateway-set 73a38db4-20f9-49ba-b855-472be1e2fd45 e0368da6-197e-4c46-bfd1-4897d61b519a # Step 6: Create floating IP from public network 'neutron floatingip-create <public network id>' $ neutron floatingip-create 1452712a-acb3-4341-a5ca-1d838eb2feb9 # Step 7: assicate the VM instance's port in private network to the newly created floating ip port ## get the port id of floating IP (in bold) $ stack@openstack-wangp11-01:~$ neutron floatingip-list +--------------------------------------+------------------+---------------------+--------------------------------------+ | id | fixed_ip_address | floating_ip_address | port_id | +--------------------------------------+------------------+---------------------+--------------------------------------+ | a2316e0b-6d72-420f-8a23-f8421160d3d4 | | 192.168.1.102 | 75353e16-5a16-452e-b420-0ada719d625c | +--------------------------------------+------------------+---------------------+--------------------------------------+ ## get the port id of private interface attached to VM instance $ stack@openstack-wangp11-01:~$ neutron port-list +--------------------------------------+------+-------------------+--------------------------------------------------------------------------------------+ | id | name | mac_address | fixed_ips | +--------------------------------------+------+-------------------+--------------------------------------------------------------------------------------+ | 16a3e39c-48ef-4d4d-bc8a-96b27da7ff3c | | fa:16:3e:c1:24:51 | {"subnet_id": "e0368da6-197e-4c46-bfd1-4897d61b519a", "ip_address": "192.168.1.100"} | | 616d03cd-df6a-465f-a23a-b6bf46e55f7a | | fa:16:3e:48:2e:7f | {"subnet_id": "fab94e2f-f7de-4bf6-8e9b-7ade0fcfad1f", "ip_address": "192.168.46.4"} | | 713e4541-7d22-4403-b828-94d714d94ec0 | | fa:16:3e:af:43:7b | {"subnet_id": "fab94e2f-f7de-4bf6-8e9b-7ade0fcfad1f", "ip_address": "192.168.46.2"} | | 75353e16-5a16-452e-b420-0ada719d625c | | fa:16:3e:23:11:db | {"subnet_id": "fab94e2f-f7de-4bf6-8e9b-7ade0fcfad1f", "ip_address": "192.168.46.3"}
# 上面红色的port是VM在private network的interface $ stack@openstack-wangp11-01:~$ neutron floatingip-associate a2316e0b-6d72-420f-8a23-f8421160d3d4 75353e16-5a16-452e-b420-0ada719d625c # Now you can use '192.168.1.102' to login to the VM instance from 192.168.1.XX network
我在最前面提到，这个Floating IP是通过namespace内的iptables规则实现的，如何查看呢？下面是在neutron node上的一个实例，192.168.1.102是Floating IP，192.168.46.3是private IP
ip netns exec qrouter-<private-network-id> iptables -t nat -S -A quantum-l3-agent-OUTPUT -d 192.168.1.102/32 -j DNAT --to-destination 192.168.46.3 -A quantum-l3-agent-PREROUTING -d 192.168.1.102/32 -j DNAT --to-destination 192.168.46.3 -A quantum-l3-agent-float-snat -s 192.168.46.3/32 -j SNAT --to-source 192.168.1.102
- 启用nameserver
现在，VM跟外界可以通过192.168.1.102进行通信了，在外界看来，VM的Ip就是192.168.1.102了，但是有一个问题，无法访问网站，也无法sudo apt-get update
必须要启用nameserver才可以访问外面的网址
# add following to /etc/resolv.conf of the VM instance to enable name service nameserver <your name sever ip> nameserver <your name server ip 2> #and run following to enable
sudo resolvconf -u
- 问题诊断
TODO
- 参考文章
https://www.mirantis.com/blog/configuring-floating-ip-addresses-networking-openstack-public-private-clouds/
https://www.rdoproject.org/Networking_in_too_much_detail
https://blogs.oracle.com/ronen/entry/running_openstack_icehouse_with_zfs
