A glance at iptables in Openstack
iptables is powerful and Openstack security group is implemented by iptables.
I took a glance at iptables in Openstack when debugging a firewall issue.
The issue
We created a VM on Computer node. The VM named test4 is in provider network and has 135.252.239.130 assigned.
cloud@Computer1:~$ nova list
+--------------------------------------+-------------+--------+------------+-------------+----------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-------------+--------+------------+-------------+----------------------+
| 7945c03b-3709-44e8-9367-2e0032891bc2 | Test server | ACTIVE | - | Running | init2=10.0.186.4 |
| d9099b07-8c2b-4633-a40f-051a05552d27 | test1 | ACTIVE | - | Running | init1=10.0.185.2 |
| fa369bbe-e963-4f0d-837e-be672c6fbb3d | test2 | ACTIVE | - | Running | init1=10.0.185.4 |
| c4e6115f-0249-4517-b852-623251abd33c | test3 | ACTIVE | - | Running | init1=10.0.185.5 |
| 0a353cbd-4075-4c96-a604-8a2ca9196a5b | test4 | ACTIVE | - | Running | ext1=135.252.239.130
The problem is VM can ping the external network but external can’t ping VM.
Debugging-1
When ping from external, do tcpdump on Computer node to make sure where the ICMP requests stop.
Check which port is used by the VM
root@Computer1:/home/cloud# neutron port-list
+--------------------------------------+------+-------------------+----------------------------------------------------------------------------------------+
| id | name | mac_address | fixed_ips |
+--------------------------------------+------+-------------------+----------------------------------------------------------------------------------------+
| 07955ef0-1e68-4e6b-ad5e-e30e33a80526 | | fa:16:3e:0a:91:25 | {"subnet_id": "7a92cc15-5eb5-4fd0-9183-5d7082e77dca", "ip_address": "10.0.185.2"} |
| 3fd3ddd3-65fd-4dba-bf88-312668563a8d | | fa:16:3e:89:b3:a6 | {"subnet_id": "c7502991-717a-43bf-a77a-587f96d7741b", "ip_address": "135.252.239.130"} |
Note:
3fd3ddd3-65will be used in iptables rules.
Double confirm the interfaces
root@Computer1:/home/cloud# ifshow | grep 3fd
qbr3fd3ddd3-65
qvb3fd3ddd3-65
qvo3fd3ddd3-65
tap3fd3ddd3-65
Ping from external and check ICMP request
root@Computer1:/home/cloud# tcpdump -n -e -i qvo3fd3ddd3-65 \(icmp or arp\) -c 2
tcpdump: WARNING: qvo3fd3ddd3-65: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on qvo3fd3ddd3-65, link-type EN10MB (Ethernet), capture size 65535 bytes
07:39:41.396735 e8:e7:32:76:b1:a2 > fa:16:3e:89:b3:a6, ethertype IPv4 (0x0800), length 98: 135.252.132.90 > 135.252.239.130: ICMP echo request, id 2860, seq 249, length 64
07:39:43.412714 e8:e7:32:76:b1:a2 > fa:16:3e:89:b3:a6, ethertype IPv4 (0x0800), length 98: 135.252.132.90 > 135.252.239.130: ICMP echo request, id 2860, seq 251, length 64
2 packets captured
2 packets received by filter
0 packets dropped by kernel
root@Computer1:/home/cloud# tcpdump -n -e -i qvb3fd3ddd3-65 \(icmp or arp\) -c 2
tcpdump: WARNING: qvb3fd3ddd3-65: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on qvb3fd3ddd3-65, link-type EN10MB (Ethernet), capture size 65535 bytes
07:39:47.444960 e8:e7:32:76:b1:a2 > fa:16:3e:89:b3:a6, ethertype IPv4 (0x0800), length 98: 135.252.132.90 > 135.252.239.130: ICMP echo request, id 2860, seq 255, length 64
07:39:48.999459 e8:e7:32:76:b1:a2 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 135.252.239.130 tell 135.252.239.129, length 46
2 packets captured
4 packets received by filter
0 packets dropped by kernel
root@Computer1:/home/cloud# tcpdump -n -e -i qbr3fd3ddd3-65 \(icmp or arp\) -c 2
tcpdump: WARNING: qbr3fd3ddd3-65: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on qbr3fd3ddd3-65, link-type EN10MB (Ethernet), capture size 65535 bytes
07:39:55.508812 e8:e7:32:76:b1:a2 > fa:16:3e:89:b3:a6, ethertype IPv4 (0x0800), length 98: 135.252.132.90 > 135.252.239.130: ICMP echo request, id 2860, seq 263, length 64
07:39:59.540855 e8:e7:32:76:b1:a2 > fa:16:3e:89:b3:a6, ethertype IPv4 (0x0800), length 98: 135.252.132.90 > 135.252.239.130: ICMP echo request, id 2860, seq 267, length 64
2 packets captured
2 packets received by filter
0 packets dropped by kernel
root@Computer1:/home/cloud# tcpdump -n -e -i tap3fd3ddd3-65 \(icmp or arp\) -c 2
tcpdump: WARNING: tap3fd3ddd3-65: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tap3fd3ddd3-65, link-type EN10MB (Ethernet), capture size 65535 bytes
^C
0 packets captured
0 packets received by filter
0 packets dropped by kernel
root@Computer1:/home/cloud#
It shows that ICMP request can reach qvo -> qvb -> qbr, and tap interface can’t receive it.
Actually the physical Ethernet interface em1 and virtual interface phy-br-ex, int-br-ex can also see IMCP request, while br-int/br-ex can’t see it.
Google tells it may be a iptables issue, so I read some iptables staff.
iptables
Here is some good for reading if you don’t know much about iptables:
- https://wiki.archlinux.org/index.php/iptables
- https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Security_Guide/sect-Security_Guide-IPTables.html
- https://www.frozentux.net/iptables-tutorial/iptables-tutorial.html
- http://rlworkman.net/howtos/iptables/cn/iptables-tutorial-cn-1.1.19.html
I was quite confused by how a packet is processed by Table and Chain. The two flows below make me clear:
A simple flow:
Note:
- The chains here mean chains in different tables, for example:
When the external packets come, firstly they are processed by PREROUTING chain. Multiple tables havePREROUTINGchain,
soPREROUTINGin the flow meansrawtable’sPREROUTING->mangletable’sPREROUTING->nattable’sPREROUTING(Refer to the first flow)
Debugging-2
Check all iptables rules:
root@Computer1:/home/cloud# iptables-save
# Generated by iptables-save v1.4.21 on Thu Aug 28 07:20:14 2014
*nat
:PREROUTING ACCEPT [2054:290530]
:INPUT ACCEPT [496:156723]
:OUTPUT ACCEPT [14232:865358]
:POSTROUTING ACCEPT [14279:872929]
:neutron-openvswi-OUTPUT - [0:0]
:neutron-openvswi-POSTROUTING - [0:0]
:neutron-openvswi-PREROUTING - [0:0]
:neutron-openvswi-float-snat - [0:0]
:neutron-openvswi-snat - [0:0]
:neutron-postrouting-bottom - [0:0]
-A PREROUTING -j neutron-openvswi-PREROUTING
-A OUTPUT -j neutron-openvswi-OUTPUT
-A POSTROUTING -j neutron-openvswi-POSTROUTING
-A POSTROUTING -j neutron-postrouting-bottom
-A POSTROUTING -s 192.168.122.0/24 -d 224.0.0.0/24 -j RETURN
-A POSTROUTING -s 192.168.122.0/24 -d 255.255.255.255/32 -j RETURN
-A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535
-A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p udp -j MASQUERADE --to-ports 1024-65535
-A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -j MASQUERADE
-A neutron-openvswi-snat -j neutron-openvswi-float-snat
-A neutron-postrouting-bottom -j neutron-openvswi-snat
COMMIT
# Completed on Thu Aug 28 07:20:14 2014
# Generated by iptables-save v1.4.21 on Thu Aug 28 07:20:14 2014
*mangle
:PREROUTING ACCEPT [1346442:407907439]
:INPUT ACCEPT [1337704:407160580]
:FORWARD ACCEPT [8785:761919]
:OUTPUT ACCEPT [1482835:433078894]
:POSTROUTING ACCEPT [1487277:433474803]
-A POSTROUTING -o virbr0 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
COMMIT
# Completed on Thu Aug 28 07:20:14 2014
# Generated by iptables-save v1.4.21 on Thu Aug 28 07:20:14 2014
*filter
:INPUT ACCEPT [204814:67869613]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [188631:59197376]
:neutron-filter-top - [0:0]
:neutron-openvswi-FORWARD - [0:0]
:neutron-openvswi-INPUT - [0:0]
:neutron-openvswi-OUTPUT - [0:0]
:neutron-openvswi-i3fd3ddd3-6 - [0:0]
:neutron-openvswi-i691b4fad-5 - [0:0]
:neutron-openvswi-idfac02b1-1 - [0:0]
:neutron-openvswi-local - [0:0]
:neutron-openvswi-o3fd3ddd3-6 - [0:0]
:neutron-openvswi-o691b4fad-5 - [0:0]
:neutron-openvswi-odfac02b1-1 - [0:0]
:neutron-openvswi-s3fd3ddd3-6 - [0:0]
:neutron-openvswi-s691b4fad-5 - [0:0]
:neutron-openvswi-sdfac02b1-1 - [0:0]
:neutron-openvswi-sg-chain - [0:0]
:neutron-openvswi-sg-fallback - [0:0]
-A INPUT -j neutron-openvswi-INPUT
-A INPUT -i virbr0 -p udp -m udp --dport 53 -j ACCEPT
-A INPUT -i virbr0 -p tcp -m tcp --dport 53 -j ACCEPT
-A INPUT -i virbr0 -p udp -m udp --dport 67 -j ACCEPT
-A INPUT -i virbr0 -p tcp -m tcp --dport 67 -j ACCEPT
-A FORWARD -j neutron-filter-top
-A FORWARD -j neutron-openvswi-FORWARD
-A FORWARD -d 192.168.122.0/24 -o virbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -s 192.168.122.0/24 -i virbr0 -j ACCEPT
-A FORWARD -i virbr0 -o virbr0 -j ACCEPT
-A FORWARD -o virbr0 -j REJECT --reject-with icmp-port-unreachable
-A FORWARD -i virbr0 -j REJECT --reject-with icmp-port-unreachable
-A OUTPUT -j neutron-filter-top
-A OUTPUT -j neutron-openvswi-OUTPUT
-A OUTPUT -o virbr0 -p udp -m udp --dport 68 -j ACCEPT
-A neutron-filter-top -j neutron-openvswi-local
-A neutron-openvswi-FORWARD -m physdev --physdev-out tap3fd3ddd3-65 --physdev-is-bridged -j neutron-openvswi-sg-chain
-A neutron-openvswi-FORWARD -m physdev --physdev-in tap3fd3ddd3-65 --physdev-is-bridged -j neutron-openvswi-sg-chain
-A neutron-openvswi-FORWARD -m physdev --physdev-out tapdfac02b1-15 --physdev-is-bridged -j neutron-openvswi-sg-chain
-A neutron-openvswi-FORWARD -m physdev --physdev-in tapdfac02b1-15 --physdev-is-bridged -j neutron-openvswi-sg-chain
-A neutron-openvswi-FORWARD -m physdev --physdev-out tap691b4fad-5d --physdev-is-bridged -j neutron-openvswi-sg-chain
-A neutron-openvswi-FORWARD -m physdev --physdev-in tap691b4fad-5d --physdev-is-bridged -j neutron-openvswi-sg-chain
-A neutron-openvswi-INPUT -m physdev --physdev-in tap3fd3ddd3-65 --physdev-is-bridged -j neutron-openvswi-o3fd3ddd3-6
-A neutron-openvswi-INPUT -m physdev --physdev-in tapdfac02b1-15 --physdev-is-bridged -j neutron-openvswi-odfac02b1-1
-A neutron-openvswi-INPUT -m physdev --physdev-in tap691b4fad-5d --physdev-is-bridged -j neutron-openvswi-o691b4fad-5
-A neutron-openvswi-i3fd3ddd3-6 -m state --state INVALID -j DROP
-A neutron-openvswi-i3fd3ddd3-6 -m state --state RELATED,ESTABLISHED -j RETURN
-A neutron-openvswi-i3fd3ddd3-6 -s 10.0.185.2/32 -j RETURN
-A neutron-openvswi-i3fd3ddd3-6 -s 10.0.185.6/32 -j RETURN
-A neutron-openvswi-i3fd3ddd3-6 -s 10.0.186.3/32 -j RETURN
-A neutron-openvswi-i3fd3ddd3-6 -s 10.0.186.4/32 -j RETURN
-A neutron-openvswi-i3fd3ddd3-6 -s 10.0.185.4/32 -j RETURN
-A neutron-openvswi-i3fd3ddd3-6 -s 10.0.185.5/32 -j RETURN
-A neutron-openvswi-i3fd3ddd3-6 -s 135.252.239.131/32 -p udp -m udp --sport 67 --dport 68 -j RETURN
-A neutron-openvswi-i3fd3ddd3-6 -j neutron-openvswi-sg-fallback
-A neutron-openvswi-i691b4fad-5 -m state --state INVALID -j DROP
-A neutron-openvswi-i691b4fad-5 -m state --state RELATED,ESTABLISHED -j RETURN
-A neutron-openvswi-i691b4fad-5 -s 10.0.185.2/32 -j RETURN
-A neutron-openvswi-i691b4fad-5 -s 135.252.239.130/32 -j RETURN
-A neutron-openvswi-i691b4fad-5 -s 10.0.185.6/32 -j RETURN
-A neutron-openvswi-i691b4fad-5 -s 10.0.186.4/32 -j RETURN
-A neutron-openvswi-i691b4fad-5 -s 10.0.185.4/32 -j RETURN
-A neutron-openvswi-i691b4fad-5 -s 10.0.185.5/32 -j RETURN
-A neutron-openvswi-i691b4fad-5 -s 10.0.186.2/32 -p udp -m udp --sport 67 --dport 68 -j RETURN
-A neutron-openvswi-i691b4fad-5 -j neutron-openvswi-sg-fallback
-A neutron-openvswi-idfac02b1-1 -m state --state INVALID -j DROP
-A neutron-openvswi-idfac02b1-1 -m state --state RELATED,ESTABLISHED -j RETURN
-A neutron-openvswi-idfac02b1-1 -s 10.0.185.2/32 -j RETURN
-A neutron-openvswi-idfac02b1-1 -s 135.252.239.130/32 -j RETURN
-A neutron-openvswi-idfac02b1-1 -s 10.0.185.6/32 -j RETURN
-A neutron-openvswi-idfac02b1-1 -s 10.0.186.3/32 -j RETURN
-A neutron-openvswi-idfac02b1-1 -s 10.0.186.4/32 -j RETURN
-A neutron-openvswi-idfac02b1-1 -s 10.0.185.4/32 -j RETURN
-A neutron-openvswi-idfac02b1-1 -s 10.0.185.3/32 -p udp -m udp --sport 67 --dport 68 -j RETURN
-A neutron-openvswi-idfac02b1-1 -j neutron-openvswi-sg-fallback
-A neutron-openvswi-o3fd3ddd3-6 -p udp -m udp --sport 68 --dport 67 -j RETURN
-A neutron-openvswi-o3fd3ddd3-6 -j neutron-openvswi-s3fd3ddd3-6
-A neutron-openvswi-o3fd3ddd3-6 -p udp -m udp --sport 67 --dport 68 -j DROP
-A neutron-openvswi-o3fd3ddd3-6 -m state --state INVALID -j DROP
-A neutron-openvswi-o3fd3ddd3-6 -m state --state RELATED,ESTABLISHED -j RETURN
-A neutron-openvswi-o3fd3ddd3-6 -j RETURN
-A neutron-openvswi-o3fd3ddd3-6 -j neutron-openvswi-sg-fallback
-A neutron-openvswi-o691b4fad-5 -p udp -m udp --sport 68 --dport 67 -j RETURN
-A neutron-openvswi-o691b4fad-5 -j neutron-openvswi-s691b4fad-5
-A neutron-openvswi-o691b4fad-5 -p udp -m udp --sport 67 --dport 68 -j DROP
-A neutron-openvswi-o691b4fad-5 -m state --state INVALID -j DROP
-A neutron-openvswi-o691b4fad-5 -m state --state RELATED,ESTABLISHED -j RETURN
-A neutron-openvswi-o691b4fad-5 -j RETURN
-A neutron-openvswi-o691b4fad-5 -j neutron-openvswi-sg-fallback
-A neutron-openvswi-odfac02b1-1 -p udp -m udp --sport 68 --dport 67 -j RETURN
-A neutron-openvswi-odfac02b1-1 -j neutron-openvswi-sdfac02b1-1
-A neutron-openvswi-odfac02b1-1 -p udp -m udp --sport 67 --dport 68 -j DROP
-A neutron-openvswi-odfac02b1-1 -m state --state INVALID -j DROP
-A neutron-openvswi-odfac02b1-1 -m state --state RELATED,ESTABLISHED -j RETURN
-A neutron-openvswi-odfac02b1-1 -j RETURN
-A neutron-openvswi-odfac02b1-1 -j neutron-openvswi-sg-fallback
-A neutron-openvswi-s3fd3ddd3-6 -s 135.252.239.130/32 -m mac --mac-source FA:16:3E:89:B3:A6 -j RETURN
-A neutron-openvswi-s3fd3ddd3-6 -j DROP
-A neutron-openvswi-s691b4fad-5 -s 10.0.186.3/32 -m mac --mac-source FA:16:3E:03:2F:A1 -j RETURN
-A neutron-openvswi-s691b4fad-5 -j DROP
-A neutron-openvswi-sdfac02b1-1 -s 10.0.185.5/32 -m mac --mac-source FA:16:3E:87:D7:82 -j RETURN
-A neutron-openvswi-sdfac02b1-1 -j DROP
-A neutron-openvswi-sg-chain -m physdev --physdev-out tap3fd3ddd3-65 --physdev-is-bridged -j neutron-openvswi-i3fd3ddd3-6
-A neutron-openvswi-sg-chain -m physdev --physdev-in tap3fd3ddd3-65 --physdev-is-bridged -j neutron-openvswi-o3fd3ddd3-6
-A neutron-openvswi-sg-chain -m physdev --physdev-out tapdfac02b1-15 --physdev-is-bridged -j neutron-openvswi-idfac02b1-1
-A neutron-openvswi-sg-chain -m physdev --physdev-in tapdfac02b1-15 --physdev-is-bridged -j neutron-openvswi-odfac02b1-1
-A neutron-openvswi-sg-chain -m physdev --physdev-out tap691b4fad-5d --physdev-is-bridged -j neutron-openvswi-i691b4fad-5
-A neutron-openvswi-sg-chain -m physdev --physdev-in tap691b4fad-5d --physdev-is-bridged -j neutron-openvswi-o691b4fad-5
-A neutron-openvswi-sg-chain -j ACCEPT
-A neutron-openvswi-sg-fallback -j DROP
COMMIT
Note:
- By default, Neutron creates chains in nat/filter table. Most chain names are like
neutron-openvswi-XXXX Each VM (or Neutron port) has its own chain. You can see the relationship in the blue color.
root@Computer1:/home/cloud# neutron port-list +--------------------------------------+--------------------+-------------------+----------------------------------------------------------------------------------------+ | id | name | mac_address | fixed_ips | +--------------------------------------+--------------------+-------------------+----------------------------------------------------------------------------------------+ | 3fd3ddd3-65fd-4dba-bf88-312668563a8d | | fa:16:3e:89:b3:a6 | {"subnet_id": "c7502991-717a-43bf-a77a-587f96d7741b", "ip_address": "135.252.239.130"} |
Let’s look into details:
When ICMP request comes into Computer node through the physical Ethernet interface em1, the host OS decides to go through the FORWARD chain.
Here are the rules for FORWARD chain:
-A FORWARD -j neutron-filter-top
-A FORWARD -j neutron-openvswi-FORWARD
-A FORWARD -d 192.168.122.0/24 -o virbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -s 192.168.122.0/24 -i virbr0 -j ACCEPT
-A FORWARD -i virbr0 -o virbr0 -j ACCEPT
-A FORWARD -o virbr0 -j REJECT --reject-with icmp-port-unreachable
-A FORWARD -i virbr0 -j REJECT --reject-with icmp-port-unreachable
Since iptables rules are in order, I will go through one by one.
-A FORWARD -j neutron-filter-top
root@Computer1:/home/cloud# iptables --line-numbers -vnL neutron-filter-top
Chain neutron-filter-top (2 references)
num pkts bytes target prot opt in out source destination
11557K 452M neutron-openvswi-local all -- * * 0.0.0.0/00.0.0.0/0
root@Computer1:/home/cloud# iptables --line-numbers -vnL neutron-openvswi-local
Chain neutron-openvswi-local (1 references)
num pkts bytes target prot opt in out source destination
So nothing is done.
-A FORWARD -j neutron-openvswi-FORWARD
root@Computer1:/home/cloud# iptables --line-numbers -vnL neutron-openvswi-FORWARD
Chain neutron-openvswi-FORWARD (1 references)
num pkts bytes target prot opt in out source destination
1 7698 644K neutron-openvswi-sg-chain all -- * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out tap3fd3ddd3-65 --physdev-is-bridged
2 2265 180K neutron-openvswi-sg-chain all -- * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in tap3fd3ddd3-65 --physdev-is-bridged
3 122 24729 neutron-openvswi-sg-chain all -- * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out tapdfac02b1-15 --physdev-is-bridged
4 94 18063 neutron-openvswi-sg-chain all -- * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in tapdfac02b1-15 --physdev-is-bridged
5 81 8828 neutron-openvswi-sg-chain all -- * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out tap691b4fad-5d --physdev-is-bridged
6 113 10634 neutron-openvswi-sg-chain all -- * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in tap691b4fad-5d --physdev-is-bridged
root@Computer1:/home/cloud# iptables --line-numbers -vnL neutron-openvswi-sg-chain
Chain neutron-openvswi-sg-chain (6 references)
num pkts bytes target prot opt in out source destination
1 7698 644K neutron-openvswi-i3fd3ddd3-6 all -- * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out tap3fd3ddd3-65 --physdev-is-bridged
2 2265 180K neutron-openvswi-o3fd3ddd3-6 all -- * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in tap3fd3ddd3-65 --physdev-is-bridged
3 122 24729 neutron-openvswi-idfac02b1-1 all -- * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out tapdfac02b1-15 --physdev-is-bridged
4 94 18063 neutron-openvswi-odfac02b1-1 all -- * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in tapdfac02b1-15 --physdev-is-bridged
5 81 8828 neutron-openvswi-i691b4fad-5 all -- * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out tap691b4fad-5d --physdev-is-bridged
6 113 10634 neutron-openvswi-o691b4fad-5 all -- * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in tap691b4fad-5d --physdev-is-bridged
7 4654 411K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0
At the incoming side, it matches this rule:
root@Computer1:/home/cloud# iptables --line-numbers -vnL neutron-openvswi-i3fd3ddd3-6
Chain neutron-openvswi-i3fd3ddd3-6 (1 references)
num pkts bytes target prot opt in out source destination
1 0 0 DROP all -- * * 0.0.0.0/00.0.0.0/0state INVALID
2 907 79632 RETURN all -- * * 0.0.0.0/00.0.0.0/0state RELATED,ESTABLISHED
3 0 0 RETURN all -- * * 10.0.185.2 0.0.0.0/0
4 0 0 RETURN all -- * * 10.0.185.6 0.0.0.0/0
5 0 0 RETURN all -- * * 10.0.186.3 0.0.0.0/0
6 0 0 RETURN all -- * * 10.0.185.4 0.0.0.0/0
7 0 0 RETURN all -- * * 10.0.185.5 0.0.0.0/0
8 5 1760 RETURN udp -- * * 135.252.239.131 0.0.0.0/0udp spt:67 dpt:68
9 5767 483K neutron-openvswi-sg-fallback all -- * * 0.0.0.0/00.0.0.0/0
So ICMP packet go through Rule 1~9, and get dropped by Rule 9:
root@Computer1:/home/cloud# iptables --line-numbers -vnL neutron-openvswi-sg-fallback
Chain neutron-openvswi-sg-fallback (12 references)
num pkts bytes target prot opt in out source destination
1 5840 498K DROP all -- * * 0.0.0.0/0 0.0.0.0/0
At this point, it is clear ICMP request is dropped by iptables.
The workaround is to create a new rule:
iptables -I neutron-openvswi-i3fd3ddd3-6 -p icmp -j RETURN
Note:
- It only allows ICMP protocol
After that, neutron-openvswi-i3fd3ddd3-6 chain is changed:
root@Computer1:/home/cloud# iptables -I neutron-openvswi-i3fd3ddd3-6 -p icmp -j RETURN
root@Computer1:/home/cloud# iptables --line-numbers -vnL neutron-openvswi-i3fd3ddd3-6
Chain neutron-openvswi-i3fd3ddd3-6 (1 references)
num pkts bytes target prot opt in out source destination
1 0 0 RETURN icmp -- * * 0.0.0.0/00.0.0.0/0
2 0 0 DROP all -- * * 0.0.0.0/00.0.0.0/0state INVALID
3 907 79632 RETURN all -- * * 0.0.0.0/00.0.0.0/0state RELATED,ESTABLISHED
4 0 0 RETURN all -- * * 135.252.239.231 0.0.0.0/0
5 0 0 RETURN all -- * * 10.0.185.2 0.0.0.0/0
6 0 0 RETURN all -- * * 10.0.185.6 0.0.0.0/0
7 0 0 RETURN all -- * * 10.0.185.4 0.0.0.0/0
8 0 0 RETURN all -- * * 10.0.185.5 0.0.0.0/0
9 5 1760 RETURN udp -- * * 135.252.239.131 0.0.0.0/0udp spt:67 dpt:68
10 5789 489K neutron-openvswi-sg-fallback all -- * * 0.0.0.0/00.0.0.0/0
Ping is successful.
root@Computer1:/home/cloud# tcpdump -n -v -e -i em1 \(icmp or arp\)
tcpdump: WARNING: em1: no IPv4 address assigned
tcpdump: listening on em1, link-type EN10MB (Ethernet), capture size 65535 bytes
08:34:23.799114 e8:e7:32:76:b1:a2 > fa:16:3e:89:b3:a6, ethertype 802.1Q (0x8100), length 102: vlan 102, p 0, ethertype IPv4, (tos 0x0, ttl 59, id 26803, offset 0, flags [DF], proto ICMP (1), length 84)
135.252.132.90 > 135.252.239.130: ICMP echo request, id 2895, seq 4, length 64
08:34:23.799456 fa:16:3e:89:b3:a6 > e8:e7:32:76:b1:a2, ethertype 802.1Q (0x8100), length 102: vlan 102, p 0, ethertype IPv4, (tos 0x0, ttl 64, id 21572, offset 0, flags [none], proto ICMP (1), length 84)
135.252.239.130 > 135.252.132.90: ICMP echo reply, id 2895, seq 4, length 64
In fact, instead of manipulate iptables, a better way is to use Security group in Openstack.
