CloudStack Advanced Networking With GRE SDN Tunnels

https://shankerbalan.net/blog/cloudstack-advanced-networking-with-gre-sdn-tunnels/

Update (20140124): OpenVswitch integration required for GRE SDN networking is currently broken in Apache CloudStack 4.2.x (and master branch) according to this.

CloudStack has two types of Zones – Basic zones and Advanced Zones.

A Basic zones is a flat network where SecurityGroups are used for traffic isolation while Advanced Zones can use VLAN, GRE or STT based isolation. The maximum VLANs possible per zone is 4096 while the actual number depends on the L3 switching device’s capabilities. Since every guest tenant is allocated one VLAN ID, the maximum tenants supported would be 4K tenants within the same zone. If there is a requirement to support more than 4K customers within the same zone, SDN technologies like GRE and STT can be used.

The GRE SDN feature seems to be most mature with XenServer hypervisor than KVM at this time. To use GRE isolation, the additional steps are roughly as below:

Update global setting sdn.ovs.controller to true

Update global setting sdn.ovs.controller.default.label to match your XenServer’s physical guest interface’s label. Mine’s labeled as GUEST

[root@vXen-2-1 ~]# xe network-list name-label=GUEST 
uuid ( RO)                : 41c845e6-4110-5d36-9f7e-b85b4909600d
          name-label ( RW): GUEST
    name-description ( RW): 
              bridge ( RO): xenbr2

Create an Advanced Zone with isolation set to GRE instead of VLAN for the guest network
Add VLAN range for the Guest. Please note that these are not actuallyVLAN ranges, but just a free range of keys used by CloudStack to create the GRE tunnels. So a range of 100 – 110 will support 11 tenants over GRE.
Assign IP addresses to the XenServer physical guest interface. I have used 192.168.65.0/24 across all the XenServer hosts

Turn on the Zone once these settings are in place. On creation of a new instance, GRE tunnels get created per tenant. Sample ovs-vsctl, ovs-ofctland tcpdump output below. All VMs share the same tunnel per tenant. In this case, its tunnel with GRE key “103”.

ovs-vsctl

[root@vXen-2-1 ~]# ovs-vsctl show
    Bridge "xapi1"
        fail_mode: standalone
        Port "xapi1"
            Interface "xapi1"
                type: internal
        Port "vif4.0"
            Interface "vif4.0"
        Port "vif3.0"
            Interface "vif3.0"
        Port "t103-1-3"
            Interface "t103-1-3"
                type: gre
                options: {key="103", remote_ip="192.168.65.246"}
    ovs_version: "1.4.2"

ovs-ofctl

[root@vXen-2-1 ~]# ovs-ofctl show xapi1
OFPT_FEATURES_REPLY (xid=0x1): ver:0x1, dpid:00008219499edf3d
n_tables:255, n_buffers:256
features: capabilities:0xc7, actions:0xfff
 1(vif3.0): addr:fe:ff:ff:ff:ff:ff
     config:     0
     state:      0
 2(t103-1-3): addr:06:19:81:4a:ce:38
     config:     0
     state:      0
 3(vif4.0): addr:fe:ff:ff:ff:ff:ff
     config:     0
     state:      0
 LOCAL(xapi1): addr:4a:68:9f:b2:cc:46
     config:     0
     state:      0
OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0

tcpdump

[root@vXen-2-1 log]# tcpdump -n -i xenbr2
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on xenbr2, link-type EN10MB (Ethernet), capture size 65535 bytes
01:36:34.580322 IP 192.168.65.247 > 192.168.65.246: GREv0, key=0x67, length 82: IP 10.1.1.63.38795 > 10.1.1.1.domain: 1255+ A? VM2.xen2.local. (32)
01:36:34.583055 IP 192.168.65.246 > 192.168.65.247: GREv0, key=0x67, length 98: IP 10.1.1.1.domain > 10.1.1.63.38795: 1255* 1/0/0 A 10.1.1.196 (48)
01:36:34.590466 IP 192.168.65.246 > 192.168.65.247: GREv0, key=0x67, length 119: IP 10.1.1.1.domain > 10.1.1.63.60399: 40706* 1/0/0 PTR VM2.xen2.local. (69)
^C
3 packets captured
3 packets received by filter
0 packets dropped by kernel

Virtual Router Network Interfaces

VM Instance Network Interface

/var/log/ovstunnel.log

2013-09-16 01:07:03    DEBUG [root] Entering create_tunnel
2013-09-16 01:07:03    DEBUG [root] Executing:['/usr/bin/ovs-vsctl', '--timeout=30', 'wait-until', 'bridge', 'xapi1', '--', 'get', 'bridge', 'xapi1', 'name']
2013-09-16 01:07:03    DEBUG [root] bridge xapi1 for creating tunnel - VERIFIED
2013-09-16 01:07:03    DEBUG [root] Executing:['/usr/bin/ovs-vsctl', 'add-port', 'xapi1', 't103-3-1', '--', 'set', 'interface', 't103-3-1', 'type=gre', 'options:key=103', 'options:remote_ip=192.168.65.247']
2013-09-16 01:07:03    DEBUG [root] Executing:['/usr/bin/ovs-vsctl', 'get', 'port', 't103-3-1', 'interfaces']
2013-09-16 01:07:03    DEBUG [root] Executing:['/usr/bin/ovs-vsctl', 'get', 'interface', 'af37904a-6f7b-4391-9b84-61bd3dba0877', 'options:key']
2013-09-16 01:07:03    DEBUG [root] Executing:['/usr/bin/ovs-vsctl', 'get', 'interface', 'af37904a-6f7b-4391-9b84-61bd3dba0877', 'options:remote_ip']
2013-09-16 01:07:03    DEBUG [root] Tunnel interface validated:['/usr/bin/ovs-vsctl', 'get', 'interface', 'af37904a-6f7b-4391-9b84-61bd3dba0877', 'options:remote_ip']
2013-09-16 01:07:03    DEBUG [root] Executing:['/usr/bin/ovs-vsctl', 'get', 'interface', 'af37904a-6f7b-4391-9b84-61bd3dba0877', 'ofport']
2013-09-16 01:07:03    DEBUG [root] Executing:['/usr/bin/ovs-ofctl', 'add-flow', 'xapi1', 'hard_timeout=0,idle_timeout=0,priority=1000,in_port=4,dl_dst=ff:ff:ff:ff:ff:ff,actions=drop']
2013-09-16 01:07:03    DEBUG [root] Executing:['/usr/bin/ovs-ofctl', 'add-flow', 'xapi1', 'hard_timeout=0,idle_timeout=0,priority=1000,in_port=4,ip,nw_dst=224.0.0.0/24,actions=drop']
2013-09-16 01:07:03    DEBUG [root] Broadcast drop rules added

For further reading, please seehttps://cwiki.apache.org/confluence/display/CLOUDSTACK/Enhancements+to+GRE-based+SDN+overlay

posted on 2016-09-03 11:44 kira+ 阅读(184) 评论(0) 编辑收藏举报