[转] OpenStack neutron floatingips 与 iptables 深入分析
转自 http://blog.csdn.net/starean/article/details/16860819
1. 简介neutron-l3-agent
OpenStack neutron-l3-agent 主要负责实现网络三层协议，为虚拟机完成SNAT,DNAT等地址的转换与伪装，提供安全弹性隔离的云网络环境，
下面详细叙述了OpenStack如何使用iptables链与规则完成复杂的neutron-l3-agent 的网络地址转换(NAT)功能，虚拟机floating ip与fixed ip绑定的工作原理。
2. iptables 简介
２.1 iptables 链拓扑结构
2.2 iptables 表结构
Table filter:
Chain INPUT
Chain FORWARDChain OUTPUT
filter 表用于一般的信息包过滤，它包含 INPUT 、 OUTPUT 和 FORWARD 链。
Table nat:
Chain PREROUTING
Chain OUTPUT
Chain POSTROUTING
PREROUTING 链由指定信息包一到达防火墙就改变它们的规则所组成，而 POSTROUTING 链由指定正当信息包打算离开防火墙时改变它们的规则所组成。
3. iptables command
# 添加一条规则到 INPUT 链的末尾，ACCEPT 来自源地址 10.9.1.141 的包
- [root@xianghui-10-9-1-141 ~]# iptables -A INPUT -s 10.9.1.141 -j ACCEPT
#允许protocol为TCP 、 UDP 、 ICMP 的包通过
- [root@xianghui-10-9-1-141 ~]# iptables -A INPUT -p TCP, UDP
# 从INPUT链中删除掉规则“Drop 到端口80的包”
- [root@xianghui-10-9-1-141 ~]# iptables -D INPUT --dport 80 -j DROP
- [root@xianghui-10-9-1-141 ~]# iptables -P INPUT DROP
# 创建一个新链new-chain
- [root@xianghui-10-9-1-141 ~]# iptables -N new-chain
# 删除Table filter 中的所有规则
- [root@xianghui-10-9-1-141 ~]# iptables -F
# 列出INPUT链中的所有规则
- [root@xianghui-10-9-1-141 ~]# iptables -L INPUT
# 删除链
- [root@xianghui-10-9-1-141 ~]# iptables -X
4. 配置neutron-l3-agent
- [root@xianghui-10-9-1-141 ~]# neutron router-create router1
- +--------------------------------------+---------+-----------------------+
- | id | name | external_gateway_info |
- +--------------------------------------+---------+-----------------------+
- |c36b384e-b1f5-45e5-bb4f-c3ed32885142 | router1 | null |
- +--------------------------------------+---------+-----------------------+
- [root@xianghui-10-9-1-141 ~]# vi /etc/neutron/l3_agent.ini
- interface_driver = neutron.agent.linux.interface.OVSInterfaceDriver
- # OS is RHEL6.4, not support namespace
- use_namespaces = False
- # This is done by setting the specific router_id.
- router_id = c36b384e-b1f5-45e5-bb4f-c3ed32885142
- # Name of bridge used for external network traffic. This should be set to
- # empty value for the linux bridge
- external_network_bridge = br-eth1
- [root@xianghui-10-9-1-141 ~]# service neutron-l3-agent restart
启用转发功能
- [root@xianghui-10-9-1-141 ~]# echo 1 > /proc/sys/net/ipv4/ip_forward
5. neutron 利用iptables 实现 NAT 原理
iptables 中neutron l3 agent自定义的链:neutron-l3-agent-PREROUTING
neutron-l3-agent-OUTPUT
neutron-l3-agent-POSTROUTING
创建外部网络（分配floatingip）
- [root@xianghui-10-9-1-141 ~]# neutron net-create ext_net --router:external=True
- +---------------------------+--------------------------------------+
- | Field | Value |
- +---------------------------+--------------------------------------+
- | admin_state_up | True |
- | id | 2d72d81b-cf09-459e-87fb-a50fa0e8730a |
- | name | ext_net |
- | provider:network_type | vlan |
- | provider:physical_network | physnet1 |
- | provider:segmentation_id | 1000 |
- | router:external | True |
- | shared | False |
- | status | ACTIVE |
- | subnets | e1932e73-1e4b-4f87-9ebf-758a757e20ef |
- | tenant_id | b21a96e16c3c438caab4a27a1f58a5b8 |
- +---------------------------+--------------------------------------+
- [root@oc2603148815 cfn]# subnet-create ext_net --allocation-pool start=192.168.12.10,end=192.168.12.50 --gateway 192.168.12.1 192.168.12.0/24 --enable_dhcp=False
- +------------------+----------------------------------------------------+
- | Field | Value |
- +------------------+----------------------------------------------------+
- | allocation_pools | {"start": "192.168.12.10", "end": "192.168.12.50"} |
- | cidr | 192.168.12.0/24 |
- | dns_nameservers | |
- | enable_dhcp | False |
- | gateway_ip | 192.168.12.1 |
- | host_routes | |
- | id | e1932e73-1e4b-4f87-9ebf-758a757e20ef |
- | ip_version | 4 |
- | name | |
- | network_id | 2d72d81b-cf09-459e-87fb-a50fa0e8730a |
- | tenant_id | b21a96e16c3c438caab4a27a1f58a5b8 |
- +------------------+----------------------------------------------------+
- [root@oc2603148815 cfn]# neutron net-create vlan-70 --provider:network_type vlan --provider:physical_network physnet1 --provider:segmentation_id 16
- +---------------------------+--------------------------------------+
- | Field | Value |
- +---------------------------+--------------------------------------+
- | admin_state_up | True |
- | id | 793a95b7-cf1f-4bde-b7b8-5a9a2e552fae |
- | name | vlan-70 |
- | provider:network_type | vlan |
- | provider:physical_network | physnet1 |
- | provider:segmentation_id | 16 |
- | router:external | False |
- | shared | False |
- | status | ACTIVE |
- | subnets | f542941d-5d53-45e4-85d0-944e030c2bcc |
- | tenant_id | b21a96e16c3c438caab4a27a1f58a5b8 |
- +---------------------------+--------------------------------------+
- [root@oc2603148815 cfn]# neutron subnet-create vlan-70 70.0.0.0/24
- +------------------+--------------------------------------------+
- | Field | Value |
- +------------------+--------------------------------------------+
- | allocation_pools | {"start": "70.0.0.2", "end": "70.0.0.254"} |
- | cidr | 70.0.0.0/24 |
- | dns_nameservers | |
- | enable_dhcp | True |
- | gateway_ip | 70.0.0.1 |
- | host_routes | |
- | id | f542941d-5d53-45e4-85d0-944e030c2bcc |
- | ip_version | 4 |
- | name | |
- | network_id | 793a95b7-cf1f-4bde-b7b8-5a9a2e552fae |
- | tenant_id | b21a96e16c3c438caab4a27a1f58a5b8 |
- +------------------+--------------------------------------------+
- [root@oc2603148815 cfn]# neutron net-list
- +--------------------------------------+---------+------------------------------------------------------+
- | id | name | subnets |
- +--------------------------------------+---------+------------------------------------------------------+
- | 2d72d81b-cf09-459e-87fb-a50fa0e8730a | ext_net | e1932e73-1e4b-4f87-9ebf-758a757e20ef 192.168.12.0/24 |
- | 793a95b7-cf1f-4bde-b7b8-5a9a2e552fae | vlan-70 | f542941d-5d53-45e4-85d0-944e030c2bcc 70.0.0.0/24 |
- +--------------------------------------+---------+------------------------------------------------------+
- # neutron router-gateway-set $ROUTER_ID $EXTERNAL_NETWORK_ID
- [root@oc2603148815 cfn]# neutron router-gateway-set 06d85a01-fc42-4cde-a0f1-377f2f394a64 2d72d81b-cf09-459e-87fb-a50fa0e8730a
- # neutron router-interface-add $ROUTER_ID $SUBNET_ID
- [root@oc2603148815 cfn]# neutron router-interface-add 06d85a01-fc42-4cde-a0f1-377f2f394a64 f542941d-5d53-45e4-85d0-944e030c2bcc
经过上面的步骤后neutron-l3-agent会加入下列规则到iptables:
- -A PREROUTING -j neutron-l3-agent-PREROUTING
- -A POSTROUTING -j neutron-l3-agent-POSTROUTING
- -A POSTROUTING -j neutron-postrouting-bottom
- -A OUTPUT -j neutron-l3-agent-OUTPUT
- -A neutron-l3-agent-snat -j neutron-l3-agent-float-snat
- -A neutron-l3-agent-snat -s 70.0.0.0/24 -j SNAT --to-source 192.168.12.10
- -A neutron-postrouting-bottom -j neutron-l3-agent-snat
创建floating ip(192.168.12.11)并绑定到vm的fixed ip(选择70.0.0.3):
- [root@xianghui-10-9-1-141 ~]# neutron floatingip-create 2d72d81b-cf09-459e-87fb-a50fa0e8730a
- Created a new floatingip:
- +---------------------+--------------------------------------+
- | Field | Value |
- +---------------------+--------------------------------------+
- | fixed_ip_address | |
- | floating_ip_address | 192.168.12.11 |
- | floating_network_id | 2d72d81b-cf09-459e-87fb-a50fa0e8730a |
- | id | f8b48ab7-ea51-4f29-bc84-0ab179808dbb |
- | port_id | |
- | router_id | |
- | tenant_id | adc4e7a4effa44ffa3c6e48dd5a8555a |
- +---------------------+--------------------------------------+
找出想要被绑定的fixed ip 的port id
- [root@xianghui-10-9-1-141 ~]# neutron port-list
- +--------------------------------------+------+-------------------+--------------------------------------------------------------------------------------+
- | id | name | mac_address | fixed_ips |
- +--------------------------------------+------+-------------------+--------------------------------------------------------------------------------------+
- | 0d06055b-2f31-4d8e-b8da-e048d76a07cc | | fa:16:3e:d7:f4:19 | {"subnet_id": "5c62752f-27ba-4d38-9702-2ca17ec2741d", "ip_address": "70.0.0.3"} |
- +--------------------------------------+------+-------------------+--------------------------------------------------------------------------------------+
- [root@xianghui-10-9-1-141 ~]# neutron floatingip-associate f8b48ab7-ea51-4f29-bc84-0ab179808dbb0d06055b-2f31-4d8e-b8da-e048d76a07cc
- Associated floatingip f8b48ab7-ea51-4f29-bc84-0ab179808dbb
- [root@xianghui-10-9-1-141 ~]# neutron floatingip-list
- +--------------------------------------+------------------+---------------------+--------------------------------------+
- | id | fixed_ip_address | floating_ip_address | port_id |
- +--------------------------------------+------------------+---------------------+--------------------------------------+
- | f8b48ab7-ea51-4f29-bc84-0ab179808dbb | 70.0.0.3 | 192.168.12.11 | b0797fe6-b799-41ea-86d0-9d9bfa0b2eb9 |
- +--------------------------------------+------------------+---------------------+--------------------------------------+
经过前面步骤后，iptables会多出下面的规则， 所有目标ip是192.168.12.11的包都会被转发到ip 70.0.0.3的guest上
- -A neutron-l3-agent-OUTPUT -d 192.168.12.11/32 -j DNAT --to-destination 70.0.0.3
- -A neutron-l3-agent-PREROUTING -d 192.168.12.11/32 -j DNAT --to-destination 70.0.0.3
- -A neutron-l3-agent-float-snat -s 70.0.0.3/32 -j SNAT --to-source 192.168.12.11
6. neutron floating ip 与 fixed ip 的转换
源地址转换(SNAT)
- [root@xianghui-10-9-1-141 ~]# iptables -t nat -Aneutron-l3-agent-float-snat -s 70.0.0.6/32-j SNAT --to-source 192.168.12.100
- [root@xianghui-10-9-1-141 ~]# iptables -t nat -Aneutron-l3-agent-PREROUTING -d 192.168.12.100/32-j DNAT --to-destination 70.0.0.6
测试：（从guest 70.0.0.11上ping 192.168.12.100, 结果被转发到70.0.0.6的guest上）
- [root@xianghui-10-9-1-141 ~]# ssh ec2-user@70.0.0.11
- [ec2-user@wordpress-test-wikidatabase-jevfsmkbakch ~]$ ping 192.168.12.100
- PING 192.168.12.100 (192.168.12.100) 56(84) bytes of data.
- 64 bytes from 70.0.0.6: icmp_req=1 ttl=64 time=3.09 ms
- 64 bytes from 70.0.0.6: icmp_req=2 ttl=64 time=0.281 ms
- 64 bytes from 70.0.0.6: icmp_req=3 ttl=64 time=0.151 ms
- [root@xianghui-10-9-1-141 ~]# iptables -t nat -A POSTROUTING -j neutron-l3-agent-float-snat
- [ec2-user@wordpress-test-wikidatabase-jevfsmkbakch ~]$ ping 192.168.12.100
- PING 192.168.12.100 (192.168.12.100) 56(84) bytes of data.
- 64 bytes from 192.168.12.100: icmp_req=1 ttl=63 time=2.47 ms
- 64 bytes from 192.168.12.100: icmp_req=2 ttl=63 time=0.199 ms
- 64 bytes from 192.168.12.100: icmp_req=3 ttl=63 time=0.251 ms
7. 实例分析（ALL-IN-ONE）
7.1 虚拟机的网络拓扑
7.2 虚拟机之间用floating ip ping通
- # ping 192.168.12.100(70.0.0.6) from 70.0.0.11
- # s:70.0.0.11 d:70.0.0.6
- # prerouting -> forward -> postrouting
- [root@xianghui-10-9-1-141 ~]# iptables -A neutron-l3-agent-FORWARD -d 70.0.0.11/32 -j ACCEPT
- [root@xianghui-10-9-1-141 ~]# iptables -A neutron-l3-agent-FORWARD -d 70.0.0.6/32 -j ACCEPT
- [root@xianghui-10-9-1-141 ~]# iptables -t nat -A neutron-l3-agent-PREROUTING -d 192.168.12.100/32 -j DNAT --to-destination 70.0.0.6
7.3 虚拟机主机ping通虚拟机的floating ip
- -A OUTPUT -j neutron-l3-agent-OUTPUT
- [root@xianghui-10-9-1-141 ~]# iptables -A neutron-l3-agent-OUTPUT -d 192.168.12.100/32 -j DNAT --to-destination 70.0.0.6
【推荐】国内首个AI IDE，深度理解中文开发场景，立即下载体验Trae
【推荐】编程新体验，更懂你的AI，立即体验豆包MarsCode编程助手
【推荐】抖音旗下AI助手豆包，你的智能百科全书，全免费不限次数
【推荐】轻量又高性能的 SSH 工具 IShell：AI 加持，快人一步
· go语言实现终端里的倒计时
· 如何编写易于单元测试的代码
· 10年+ .NET Coder 心语，封装的思维：从隐藏、稳定开始理解其本质意义
· .NET Core 中如何实现缓存的预热？
· 从 HTTP 原因短语缺失研究 HTTP/2 和 HTTP/3 的设计差异
· 周边上新：园子的第一款马克杯温暖上架
· 分享 3 个 .NET 开源的文件压缩处理库，助力快速实现文件压缩解压功能！
· Ollama——大语言模型本地部署的极速利器
· DeepSeek如何颠覆传统软件测试？测试工程师会被淘汰吗？
· 使用C#创建一个MCP客户端