N5K VPC功能配置及测试

一、概述

1.1 测试目的

Nexus 5500交换机作为业界第一款统一端口交换机，同时支持ethernet、 Fibre Channel和Fibre Channel over Ethernet (FCOE)协议。本次测试的目的是为了测试Nexus 5500交换机的功能，以验证是否适合作为服务器区的汇聚交换机，同时实现服务器区的以太网汇聚和SAN存储汇聚，从而逐渐向两网融合的方向演进。.

1.2测试环境：

• 主机：1台IBM P550
• 系统：AIX
• FC HBA：emulex 4Gb double-port
• 盘柜：HDS 2300
• 网络设备：两台Nexus 5596，一台Nexus 2148，一台Nexus 2248，一台Catalyst 3560G

以下是这次测试方案的拓扑。

 

 

 

 

 

 

 

1.3配置前准备

1.3.1配置管理地址

通过Console线连接到两台Nexus 5596的管理口上，分别配置两台设备的管理IP地址在同一网段。在本例中Nexus 5596-1的管理ip是168.7.63.11，Nexus 5596-2的管理ip是168.7.63.12。如下图所示，管理站和A、B控制器正常连接。

配置内容	备注
N5596-1   interface mgmt0 ip address 168.7.63.11/25 vrf context management ip route 0.0.0.0/0 168.7.63.126	网管接口放在单独的管理VRF里，避免数据流量的干扰
N5596-2  interface mgmt0 ip address 168.7.63.12/25 vrf context management ip route 0.0.0.0/0 168.7.63.126	网管接口放在单独的管理VRF里，避免数据流量的干扰

 

1.3.2安装存储license

缺省情况下，N5K没有安装存储license。需要通过思科网站申请临时license。申请步骤如下：

配置内容	备注
以N5596-1为例   switch# show license host-id License hostid: VDH=FOX1513GHGZ copy tftp://168.7.63.33/test.lic bootflash:test.lic vrf management install license bootflash:test.lic	到思科网站www.cisco.com/go/license申请N5K的FCOE license，申请后更改license名字（不能太长）。   将license文件拷贝到flash中。由于是通过管理口拷贝，所以要加上vrf参数。 安装存储license。

 

1.3.2N5K操作系统升级

在测试前需要升级N5K的操作系统。

配置内容	备注
以N5596-1为例   copy tftp:// n5000-uk9-kickstart.5.1.3.N1.1a.bin bootflash: n5000-uk9-kickstart.5.1.3.N1.1a.bin copy tftp:// n5000-uk9.5.1.3.N1.1a.bin bootflash: n5000-uk9.5.1.3.N1.1a.bin Install all	操作系统升级包括kickstart和system两个image。   把kickstart image拷进bootflash。 将system image文件拷进bootflash 升级操作系统

 

三、vPC功能测试

3.1 测试目的

虚拟端口捆绑技术vPC(Virtual Port-Channel)是一种扩展技术，它通过将两个交换机的转发平面整合，实现接入设备采用Port-Channel同时上联两个汇聚交换机，实现双倍的聚合带宽，并消除STP blocked ports，在link/device失效下提供快速收敛。采用板卡延伸技术和vPC技术，可以有效扩展网络规模，同时减少生成树对于网络的影响。

传统端口信道通信的最大限制在于端口信道只能在两个设备之间运行。在大型网络中，设计中常常需要同时支持多个设备，来提供某些形式的硬件故障备用路径。这一备用路径的连接方式常常会导致环路，从而限制对单一路径实施端口信道技术的优势。为突破此限制，Cisco NX-OS软件平台提供一种名为虚拟端口捆绑组，或即 vPC 的技术。尽管对于与端口信道相连的设备来说，一对作为 vPC 对等终端的交换机就像是单一逻辑实体，但这两个作为逻辑端口信道终端的设备仍是两个独立设备。该环境结合了硬件冗余性和端口信道环路管理的优势。升级到一个完全基于端口信道的环路管理机制，所能获得的另一主要优势是，链路恢复速度大大加快。生成树协议从链路故障中恢复的时间大约为6秒，而完全基于虚拟端口捆绑组vPC的解决方案则有可能在不到 1 秒能完成故障恢复。

vPC为第二层网络提供大量重要优势，并借助第二层功能提供的优势，对第三层互联进行一系列改进。

在第二层网络中，能够实现以下优势：

• 通过冗余系统提高系统可用性
• 无需使用生成树协议，即能进行环路管理
• 始终提供完全系统带宽可用性
• 迅速恢复链路故障
• 为任意支持 IEEE 802.3ad 的边缘设备提供端口信道连接

 

 

本项测试的目的是验证vPC技术。

3.2 测试拓扑

将两台Nexus 2000设备与两台Nexus 5596交叉连接，验证两台Nexus 5596上是否都能认到两台Nexus 2000设备。

3.3 测试配置

配置内容	备注
N5596-1   feature vpc vpc domain 3 peer-keepalive destination 168.7.63.12 interface port-channel230 switchport mode trunk spanning-tree port type network speed 10000 vpc peer-link interface Ethernet1/3 switchport mode trunk channel-group 230 mode active interface Ethernet1/5 switchport mode trunk channel-group 230 mode active interface port-channel100 vpc 100 interface port-channel132 switchport mode fex-fabric fex associate 132 vpc 132 interface Ethernet1/7 switchport mode fex-fabric fex associate 132 channel-group 132	启用vpc   创建vPC域，分配域ID 将管理口连接用作peer keepalive link 两台N5K互联的portchannel     将互联接口用作peer link             ；将与下联设备N2148T相连的portchannel接口放到vpc域中，分配的数字应与N5596-2相同 ；增加与N2248TP相连的配置，与N5596-2上的配置基本相同     ；与N2248TP相连的接口
N5596-2     feature vpc vpc domain 3 peer-keepalive destination 168.7.63.11 interface port-channel230 switchport mode trunk spanning-tree port type network speed 10000 vpc peer-link interface Ethernet1/3 switchport mode trunk channel-group 230 mode active interface Ethernet1/5 switchport mode trunk channel-group 230 mode active interface port-channel132 vpc 132 interface port-channel100 switchport mode fex-fabric fex associate 100 vpc 100 interface Ethernet1/7 switchport mode fex-fabric fex associate 100 channel-group 100

 

3.4测试结果

配置内容	备注
N5596-1     n5596-1# show fex FEX         FEX           FEX                       FEX Number    Description      State            Model            Serial ———————————————————————— 100        FEX0100                Online      N2K-C2148T-1GE   FOX1316GD83 132        FEX0132                Online     N2K-C2248TP-1GE   SSI141904Y2       n5596-1#show run 。。。。。。。。。。。。。 interface Ethernet100/1/1 interface Ethernet100/1/2 interface Ethernet100/1/3 。。。。。。。。。。。。。 interface Ethernet132/1/1 interface Ethernet132/1/2 interface Ethernet132/1/3 。。。。。。。。。。。。。   n5596-1# show port-channel database port-channel100 Last membership update is successful 2 ports in total, 2 ports up First operational port is Ethernet1/1 Age of the port-channel is 27d:15h:30m:25s Time since last bundle is 27d:15h:31m:25s Last bundled member is Ethernet1/2 Ports:   Ethernet1/1     [on] [up] * Ethernet1/2     [on] [up]   port-channel132 Last membership update is successful 1 ports in total, 1 ports up First operational port is Ethernet1/7 Age of the port-channel is 27d:15h:30m:25s Time since last bundle is 14d:18h:01m:33s Last bundled member is Ethernet1/7 Ports:   Ethernet1/7     [on] [up] *   port-channel230 Last membership update is successful 2 ports in total, 2 ports up First operational port is Ethernet1/3 Age of the port-channel is 14d:18h:08m:51s Time since last bundle is 14d:18h:08m:46s Last bundled member is Ethernet1/5 Ports:   Ethernet1/3     [active ] [up] * Ethernet1/5     [active ] [up]   n5596-1# show vpc brief Legend: (*) – local vPC is down, forwarding via vPC peer-link   vPC domain id                   : 3 Peer status                     : peer adjacency formed ok vPC keep-alive status           : peer is alive Configuration consistency status: success Per-vlan consistency status     : success Type-2 consistency status       : success vPC role                        : secondary, operational primary Number of vPCs configured       : 100 Peer Gateway                    : Disabled Dual-active excluded VLANs      : – Graceful Consistency Check      : Enabled   vPC Peer-link status ——————————————————————— id   Port   Status Active vlans —   —-   —— ————————————————– 1    Po230  up     1,10,50,150,300   vPC status —————————————————————————- id     Port        Status Consistency Reason                     Active vlans —— ———– —— ———– ————————– ———– 100    Po100       up     success     success                    – 132    Po132       up     success     success                    – 101377 Eth100/1/1  down*  Not         Consistency Check Not      – Applicable  Performed 。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。	；两个fex都已经正常连接                 ；配置中出现了两个FEX的接口                     ；portchannel100处于正常状态，包含和N2148T相连的e1/1和e1/2两个接口               ；portchannel132处于正常状态，包含和N2248TP相连的e1/7接口             ；portchannel230处于正常状态，包含两台N5K互联的e1/3和e1/5两个接口                                                                     ；vPC 100状态正常 ；Vpc 132状态正常
N5596-2     n5596-2# show fex FEX         FEX           FEX                       FEX Number    Description      State            Model            Serial ———————————————————————— 100        FEX0100                Online      N2K-C2148T-1GE   FOX1316GD83 132        FEX0132                Online     N2K-C2248TP-1GE   SSI141904Y2       n5596-2#show run 。。。。。。。。。。。。。 interface Ethernet100/1/1 interface Ethernet100/1/2 interface Ethernet100/1/3 。。。。。。。。。。。。。 interface Ethernet132/1/1 interface Ethernet132/1/2 interface Ethernet132/1/3 。。。。。。。。。。。。。   n5596-2# show port-channel database port-channel100 Last membership update is successful 1 ports in total, 1 ports up First operational port is Ethernet1/7 Age of the port-channel is 13d:21h:39m:23s Time since last bundle is 13d:21h:40m:22s Last bundled member is Ethernet1/7 Ports:   Ethernet1/7     [on] [up] *   port-channel132 Last membership update is successful 2 ports in total, 2 ports up First operational port is Ethernet1/1 Age of the port-channel is 13d:21h:39m:23s Time since last bundle is 13d:21h:40m:22s Last bundled member is Ethernet1/2 Ports:   Ethernet1/1     [on] [up] * Ethernet1/2     [on] [up]   port-channel230 Last membership update is successful 2 ports in total, 2 ports up First operational port is Ethernet1/3 Age of the port-channel is 13d:21h:39m:23s Time since last bundle is 13d:21h:40m:22s Last bundled member is Ethernet1/5 Ports:   Ethernet1/3     [active ] [up] * Ethernet1/5     [active ] [up]   n5596-1# show vpc brief Legend: (*) – local vPC is down, forwarding via vPC peer-link   vPC domain id                   : 3 Peer status                     : peer adjacency formed ok vPC keep-alive status           : peer is alive Configuration consistency status: success Per-vlan consistency status     : success Type-2 consistency status       : success vPC role                        : primary, operational secondary Number of vPCs configured       : 100 Peer Gateway                    : Disabled Dual-active excluded VLANs      : – Graceful Consistency Check      : Enabled   vPC Peer-link status ——————————————————————— id   Port   Status Active vlans —   —-   —— ————————————————– 1    Po230  up     1,10,50,150,300   vPC status —————————————————————————- id     Port        Status Consistency Reason                     Active vlans —— ———– —— ———– ————————– ———– 100    Po100       up     success     success                    – 132    Po132       up     success     success                    – 101377 Eth100/1/1  down*  Not         Consistency Check Not      – Applicable  Performed 。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。

 

vPC功能工作正常。

四、vPC兼容性测试

4.1 测试目的

在vPC的部署场景中，接入设备的选择是一个重要的考虑因素。N2K FEX设备作为N5K设备的扩展板卡，支持vPC。但是接入设备也可能选择比较便宜的传统以太网接入交换机，例如Catalyst 2960。

本项测试的目的是验证vPC技术对于传统的以太网接入交换机同样适用。

4.2 测试拓扑

 

将Catalyst 3560G的两个SFP上联接口分别与两台Nexus 5596连接，验证Catalyst 3560G上可以实现跨机箱的链路捆绑。

4.3 测试配置

配置内容	备注
N5596-1     feature interface-vlan interface Vlan300 no shutdown management ip address 10.48.50.202/24 interface port-channel20 switchport mode trunk speed 1000 vpc 20 interface Ethernet1/9 switchport mode trunk speed 1000 channel-group 20 mode active	；支持在vlan接口上配置IP地址     ；为与接入交换机相连的portchannel口分配vpc号     ；N5K上插千兆SFP时需手动指定速率
N5596-2     feature interface-vlan interface Vlan300 no shutdown management ip address 10.48.50.132/24 interface port-channel20 switchport mode trunk speed 1000 vpc 20 interface Ethernet1/9 switchport mode trunk speed 1000 channel-group 20 mode active
Catalyst 3560G     interface GigabitEthernet0/27 switchport mode trunk channel-group 20 mode active interface GigabitEthernet0/28 switchport mode trunk channel-group 20 mode active interface port-channel20 switchport mode trunk interface Vlan300 no shutdown ip address 10.48.50.168

 

4.4测试结果

配置内容	备注
N5596-1     n5596-1# show port-channel database port-channel20 Last membership update is successful 1 ports in total, 0 ports up Age of the port-channel is 14d:17h:46m:34s Time since last bundle is 14d:17h:46m:34s Last bundled member is Ethernet1/9 Ports:   Ethernet1/9     [active ] [individual]   n5596-1# show vpc brief Legend: (*) – local vPC is down, forwarding via vPC peer-link   vPC domain id                   : 3 Peer status                     : peer adjacency formed ok vPC keep-alive status           : peer is alive Configuration consistency status: success Per-vlan consistency status     : success Type-2 consistency status       : success vPC role                        : secondary, operational primary Number of vPCs configured       : 100 Peer Gateway                    : Disabled Dual-active excluded VLANs      : – Graceful Consistency Check      : Enabled   vPC Peer-link status ——————————————————————— id   Port   Status Active vlans —   —-   —— ————————————————– 1    Po230  up     1,10,50,150,300   vPC status —————————————————————————- id     Port        Status Consistency Reason                     Active vlans —— ———– —— ———– ————————– ———– 20     Po20        up     success     success                    -。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。   n5596-1# show int po20 port-channel20 is up vPC Status: up, vPC number: 20 Hardware: Port-Channel, address: 547f.ee07.3510 (bia 547f.ee07.3510) MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec reliability 255/255, txload 1/255, rxload 1/255 Encapsulation ARPA Port mode is trunk auto-duplex, 1000 Mb/s Input flow-control is off, output flow-control is off Switchport monitor is off EtherType is 0x8100 Members in this channel: Eth1/9 Last clearing of “show interface” counters never 30 seconds input rate 2328 bits/sec, 3 packets/sec 30 seconds output rate 904 bits/sec, 1 packets/sec	；portchannel 20正常                                                                       ；vPC 20状态正常       ；portchannel20接口up
N5596-2     n5596-2# show port-channel database port-channel20 Last membership update is successful 1 ports in total, 0 ports up Age of the port-channel is 13d:21h:39m:23s Time since last bundle is 13d:21h:40m:22s Last bundled member is Ethernet1/9 Ports:   Ethernet1/9     [active ] [individual]   n5596-2# show vpc brief Legend: (*) – local vPC is down, forwarding via vPC peer-link   vPC domain id                   : 3 Peer status                     : peer adjacency formed ok vPC keep-alive status           : peer is alive Configuration consistency status: success Per-vlan consistency status     : success Type-2 consistency status       : success vPC role                        : primary, operational secondary Number of vPCs configured       : 100 Peer Gateway                    : Disabled Dual-active excluded VLANs      : – Graceful Consistency Check      : Enabled   vPC Peer-link status ——————————————————————— id   Port   Status Active vlans —   —-   —— ————————————————– 1    Po230  up     1,10,50,150,300   vPC status —————————————————————————- id     Port        Status Consistency Reason                     Active vlans —— ———– —— ———– ————————– ———– 20     Po20        up     success     success                    -。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。   n5596-2# show int po20 port-channel20 is up vPC Status: up, vPC number: 20 Hardware: Port-Channel, address: 547f.ee07.3510 (bia 547f.ee07.3510) MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec reliability 255/255, txload 1/255, rxload 1/255 Encapsulation ARPA Port mode is trunk auto-duplex, 1000 Mb/s Input flow-control is off, output flow-control is off Switchport monitor is off EtherType is 0x8100 Members in this channel: Eth1/9 Last clearing of “show interface” counters never 30 seconds input rate 2280 bits/sec, 3 packets/sec 30 seconds output rate 1008 bits/sec, 1 packets/sec
Catalyst 3560G     Show interface po20	；portchannel接口正常   ；拔掉一个上联端口，PING不丢包

 

vPC与普通以太网交换机配合工作正常。

 

五、EvPC功能测试

5.1 测试目的

EvPC即增强vPC或者双层vPC。在EvPC的部署场景中，不仅接入设备可以跨N5K进行链路捆绑，而且服务器做NIC TEAMING的时候也可以跨不同接入设备做loadbalance，而不仅仅是Active-Standby。支持EvPC的接入设备只能是N2K，而不能是普通的以太网交换机。

 

本项测试的目的是验证EvPC技术。

5.2 测试拓扑

 

将服务器IBM P550的两个以太网口分别连接到Nexus 2148T和Nexus 2248TP，配置NIC teaming为loadbalance模式，验证NIC teaming是否正常。

5.3 测试配置

配置内容	备注
N5596-1     interface port-channel3 switchport access vlan 300 interface Ethernet100/1/1 switchport access vlan 300 channel-group 3 interface Ethernet132/1/1 switchport access vlan 300 channel-group 3	；两台N5K的Portchannel号必须一样 ；EvPC不需要手动分配vPC号，系统内部会自动分配 ；对同一个FEX端口的配置，两台N5K上必须完全一样，否则FEX端口无法正常工作
N5596-2     interface port-channel3 switchport access vlan 300 interface Ethernet100/1/1 switchport access vlan 300 channel-group 3 interface Ethernet132/1/1 switchport access vlan 300 channel-group 3

 

5.4测试结果

配置内容	备注
N5596-1     n5596-1(config-if)# show int po3 port-channel3 is up vPC Status: Up, vPC number: 262146 Hardware: Port-Channel, address: 5475.d0e8.c042 (bia 5475.d0e8.c042) MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec reliability 255/255, txload 1/255, rxload 1/255 Encapsulation ARPA Port mode is access full-duplex, 1000 Mb/s Input flow-control is off, output flow-control is on Switchport monitor is off EtherType is 0x8100 Members in this channel: Eth100/1/1, Eth132/1/1 Last clearing of “show interface” counters never 30 seconds input rate 28560 bits/sec, 35 packets/sec 30 seconds output rate 28632 bits/sec, 35 packets/sec Load-Interval #2: 5 minute (300 seconds) input rate 2.78 Kbps, 3 pps; output rate 2.99 Kbps, 3 pps RX 2379 unicast packets  1 multicast packets  100 broadcast packets 2480 input packets  198316 bytes 0 jumbo packets  0 storm suppression bytes 0 runts  0 giants  0 CRC  0 no buffer 0 input error  0 short frame  0 overrun   0 underrun  0 ignored 0 watchdog  0 bad etype drop  0 bad proto drop  0 if down drop 0 input with dribble  0 input discard 0 Rx pause TX 2623 unicast packets  146511 multicast packets  2376 broadcast packets 151510 output packets  38551096 bytes 0 jumbo packets 0 output errors  0 collision  0 deferred  0 late collision 0 lost carrier  0 no carrier  0 babble 2 output discard 0 Tx pause	；vPC状态正常，vPC号自动分配
N5596-2     n5596-2# show int po3 port-channel3 is up vPC Status: Up, vPC number: 262146 Hardware: Port-Channel, address: 5475.d0e8.c042 (bia 5475.d0e8.c042) MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec reliability 255/255, txload 1/255, rxload 1/255 Encapsulation ARPA Port mode is access full-duplex, 1000 Mb/s Input flow-control is off, output flow-control is on Switchport monitor is off EtherType is 0x8100 Members in this channel: Eth100/1/1, Eth132/1/1 Last clearing of “show interface” counters never 30 seconds input rate 0 bits/sec, 0 packets/sec 30 seconds output rate 144 bits/sec, 0 packets/sec Load-Interval #2: 5 minute (300 seconds) input rate 0 bps, 0 pps; output rate 112 bps, 0 pps RX 2380 unicast packets  1 multicast packets  100 broadcast packets 2481 input packets  198380 bytes 0 jumbo packets  0 storm suppression bytes 0 runts  0 giants  0 CRC  0 no buffer 0 input error  0 short frame  0 overrun   0 underrun  0 ignored 0 watchdog  0 bad etype drop  0 bad proto drop  0 if down drop 0 input with dribble  0 input discard 0 Rx pause TX 2624 unicast packets  146527 multicast packets  2376 broadcast packets 151527 output packets  38555566 bytes 0 jumbo packets 0 output errors  0 collision  0 deferred  0 late collision 0 lost carrier  0 no carrier  0 babble 8 output discard 0 Tx pause 4 interface resets
IBM P550     将网卡地址配置为10.48.50.218，PING N5596-1的vlan 300接口地址10.48.50.132，将连接N2148T的e100/1/1接口拔掉，不丢包，证明portchannel生效。

 

EvPC功能工作正常。

 

 [dltable file=”N5K-VPC功能配置及测试.doc” size=”3M”]N5K-VPC功能配置及测试.doc[/dltable]