Sunday, September 22, 2013

Nexus vPC to Nortel Switches

I have a customer that currently has a network with a pair of Nortel Passport 8000 switches as their core and a bunch of Nortel 5510 and 5520 access switches for server and workstation connectivity. This customer decided to purchase a pair of Nexus 5596T switches to replace the Nortel 8000 switches.

In the existing environment, they are utilizing Nortel's split multi-link trunking (SMLT) for layer 2 resiliency and aggregation. I have been tasked with setup of the new Nexus switches and configuring a similar topology using Cisco vPC technology for layer 2 resiliency and aggregation.

In this example, we are going to create vPC peer-link and a single LACP port channel to a Nortel 5510 switch.



Let's start with the basic Nexus vPC configuration. I'm not going to spend a lot of time on it as it's been covered by many people.

Nexus-01
feature vpc
feature lacp

interface mgmt0
  ip address 192.168.200.253/24

vpc domain 1
  role priority 4096
  peer-keepalive destination 192.168.200.253
  peer-gateway
  auto-recovery

interface Ethernet1/31
  descriptoin vPC Peer Link
  switchport mode trunk
  spanning-tree port type network
  channel-group 1 mode active

interface Ethernet1/32
  description vPC Peer Link
  switchport mode trunk
  spanning-tree port type network
  channel-group 1 mode active

interface port-channel1
  switchport mode trunk
  spanning-tree port type network
  vpc peer-link

interface vlan2
  ip address 192.168.10.254/24
  standby 2 ip 192.168.10.1
  standby 2 priority 110
  no shutdown

Nexus-02
feature vpc
feature lacp 

interface mgmt0
  ip address 192.168.200.254/24

vpc domain 1
  role priority 8192
  peer-keepalive destination 192.168.200.254
  peer-gateway
  auto-recovery

interface Ethernet1/31
  descriptoin vPC Peer Link
  switchport mode trunk
  spanning-tree port type network
  channel-group 1 mode active

interface Ethernet1/32
  description vPC Peer Link
  switchport mode trunk
  spanning-tree port type network
  channel-group 1 mode active

interface port-channel1
  switchport mode trunk
  spanning-tree port type network
  vpc peer-link

interface vlan2
  ip address 192.168.10.253/24
  standby 2 ip 192.168.10.1
  no shutdown

And now we confirm that the vPC peer-link is up and working properly.

Nexus-01# sh vpc
Legend:
                (*) - local vPC is down, forwarding via vPC peer-link

vPC domain id                     : 1
Peer status                       : peer adjacency formed ok
vPC keep-alive status             : peer is alive
Configuration consistency status  : success
Per-vlan consistency status       : success
Type-2 consistency status         : success
vPC role                          : primary
Number of vPCs configured         : 1
Peer Gateway                      : Enabled
Peer gateway excluded VLANs     : -
Dual-active excluded VLANs        : -
Graceful Consistency Check        : Enabled
Auto-recovery status              : Enabled (timeout = 240 seconds)

vPC Peer-link status
---------------------------------------------------------------------
id   Port   Status Active vlans
--   ----   ------ --------------------------------------------------
1    Po1    up     1-2

Nexus-02# sh vpc
Legend:
                (*) - local vPC is down, forwarding via vPC peer-link

vPC domain id                     : 1
Peer status                       : peer adjacency formed ok
vPC keep-alive status             : peer is alive
Configuration consistency status  : success
Per-vlan consistency status       : success
Type-2 consistency status         : success
vPC role                          : secondary
Number of vPCs configured         : 1
Peer Gateway                      : Enabled
Peer gateway excluded VLANs     : -
Dual-active excluded VLANs        : -
Graceful Consistency Check        : Enabled
Auto-recovery status              : Enabled (timeout = 240 seconds)

vPC Peer-link status
---------------------------------------------------------------------
id   Port   Status Active vlans
--   ----   ------ --------------------------------------------------
1    Po1    up     1-2

And now that the vPC connection is up between the switches, we can configure the port-channel for the Nortel switch.
Nexus-01

interface Ethernet1/1
  description Trunk to Nortel Test
  switchport trunk allowed vlan 1-2
  switchport mode trunk
  channel-group 10 mode active

interface port-channel10
  description Trunk to Nortel Test
  no lacp graceful-convergence
  switchport trunk allowed vlan 1-2
  switchport mode trunk
  vpc 10

Nexus-02

interface Ethernet1/1
  description Trunk to Nortel Test
  switchport trunk allowed vlan 1-2
  switchport mode trunk
  channel-group 10 mode active

interface port-channel10
  description Trunk to Nortel Test
  no lacp graceful-convergence
  switchport trunk allowed vlan 1-2
  switchport mode trunk
  vpc 10

*Update: The configuration "no lacp graceful-convergence" was added to the port channel interfaces. LACP graceful convergence was introduced in version 5.1(3)N1(1) and is on by default. Cisco recommends disabling this feature on port channels connected to non NX-OS devices as it can cause the interfaces to inexplicibly enter a standby state.

And now for the multi-link trunk (MLT) configuration on the Nortel switch.
ip address switch 192.168.10.10
ip address netmask 255.255.255.0
vlan mgmt 2

vlan create 2 type port 1
vlan ports 47-48 tagging unTagPvidOnly
vlan members 1 1-48
vlan members 2 47-48
vlan ports 47-48 pvid 1

interface fastEthernet 47-48
lacp key 10
lacp timeout-time short
lacp mode active
lacp aggregation enable

Note that I left the PVID untagged for the two uplink ports to ensure compatibility with the Nexus switches since Cisco uses the concept of native VLANs on trunk ports. For this example, we are keeping it simple and using VLAN 1 for the native VLAN.

When you configure an LACP key on the Nortel switch, it assigns the unique LACP key to an MLT starting with the highest number MLT (32 in this case) and working backwards. The LACP key is local to the switch or stack and is similar to the channel-group with the Nexus switches. The MLT ID is similar to the port-channel interface on the Nexus switches.

At this point, the port-channel and MLT should be up on all the switches. Let's confirm.

Nexus-01# sh lacp neigh
Flags:  S - Device is sending Slow LACPDUs F - Device is sending Fast LACPDUs
        A - Device is in Active mode       P - Device is in Passive mode
PO10 neighbors
Partner's information
            Partner                Partner                     Partner
Port        System ID              Port Number     Age         Flags
Eth1/1      32768,0-19-69-a6-4c-0  0x2f            378         SA

            LACP Partner           Partner                     Partner
            Port Priority          Oper Key                    Port State
            32768                  0x300a                      0x3f

Nexus-02# sh lacp neigh
Flags:  S - Device is sending Slow LACPDUs F - Device is sending Fast LACPDUs
        A - Device is in Active mode       P - Device is in Passive mode
Po10 neighbors
Partner's information
            Partner                Partner                     Partner
Port        System ID              Port Number     Age         Flags
Eth1/1      32768,0-19-69-a6-4c-0  0x30            692         SA

            LACP Partner           Partner                     Partner
            Port Priority          Oper Key                    Port State
            32768                  0x300a                      0x3f

Nexus-01# sh vpc
!------Excluded for brevity---------!
vPC status
----------------------------------------------------------------------------
id     Port        Status Consistency Reason                     Active vlans
------ ----------- ------ ----------- -------------------------- -----------
10     Po10        up     success     success                    1-2

Nexus-02# sh vpc
!------Excluded for brevity---------!
vPC status
----------------------------------------------------------------------------
id     Port        Status Consistency Reason                     Active vlans
------ ----------- ------ ----------- -------------------------- -----------
10     Po10        up     success     success                    1-2

5510-48T-PWR# sh lacp port 47-48
                                  Admin Oper         Trunk Partner
Port Priority Lacp    A/I Timeout Key   Key   AggId Id    Port    Status
---- -------- ------- --- ------- ----- ----- ----- ----- ------- -------
47   32768    Active  A   Short   10    12298 8224   32    257     Active
48   32768    Active  A   Short   10    12298 8224   32    16641   Active

As you can see, LACP is up on all switches and vPC 10 is up on both Nexus switches. At this point we can confirm connectivity.
Nexus-01# ping 192.168.10.10
PING 192.168.10.10 (192.168.10.10): 56 data bytes
64 bytes from 192.168.10.10: icmp_seq=0 ttl=63 time=1.697 ms
64 bytes from 192.168.10.10: icmp_seq=1 ttl=63 time=1.305 ms
64 bytes from 192.168.10.10: icmp_seq=2 ttl=63 time=1.287 ms
64 bytes from 192.168.10.10: icmp_seq=3 ttl=63 time=1.329 ms
64 bytes from 192.168.10.10: icmp_seq=4 ttl=63 time=1.393 ms

Nexus-02# ping 192.168.10.10
PING 192.168.10.10 (192.168.10.10): 56 data bytes
64 bytes from 192.168.10.10: icmp_seq=0 ttl=62 time=9.314 ms
64 bytes from 192.168.10.10: icmp_seq=1 ttl=62 time=3.379 ms
64 bytes from 192.168.10.10: icmp_seq=2 ttl=62 time=3.349 ms
64 bytes from 192.168.10.10: icmp_seq=3 ttl=62 time=3.354 ms
64 bytes from 192.168.10.10: icmp_seq=4 ttl=62 time=3.352 ms

5510-48T-PWR>ping 192.168.10.1
Host is reachable
5510-48T-PWR>ping 192.168.10.254
Host is reachable
5510-48T-PWR>ping 192.168.10.253
Host is reachable

As we can see, ping works from each of the Nexus switches and from the Nortel switch to each Nexus switch and the shared HSRP IP.

The biggest caveat with this configuration is that you cannot change VLAN assignments on the trunk while LACP is active on the Nortel switch. If you change the allowed VLANs on either side of the trunk, it brings down the LACP neighborship until you disable and re-enable LACP on both uplinks. For this reason, I recommend you set the allowed VLANs on the port-channel on the Nexus switches to prevent the trunk from going down simply by creating a VLAN on the switch.