It's that time of year again. New Years traditionally is a time to reflect upon the year and make resolutions for the upcoming year. Like so many others, I have decided to make a blog post to document and motivate my goals for 2014. So here goes.
Professional Goals
1. Attain a better work/life balance. As is the nature of networking and working for a VAR, I find myself working after hours quite frequently. This makes it challenging to keep a healthy lifestyle outside of the workplace. This is going to be a work in progress all year, but I hope to find a good balance.
2. Blog more. I have several ideas saved up for future blog posts that I would like to write. The ultimate goal is to make it easier for younger people to enter and succeed in the field. As a side effect, it will help me learn as I further my learning and will be a reference for me in the future.
3. Attain CCNP R&S certification. I have the CCNP switch exam out of the way. I am currently studying for the routing exam and I purchased a block of Cisco Learning Lab hours. I would like pass both routing and tshoot exams in 2014.
Personal Goals
1. I would like to become more physically fit. I'm not really out of shape, but I don't work proactively to keep in shape. As I've discovered, the better I take care of myself the better I feel. I recently ordered a FitBit Force, which tracks fitness habits and sleep patterns.
2. Spend more time with family and friends. My family all lives close by, but I don't see them as often as I would like, maybe once or twice a month. My brother has three kids that I love to death and would love to see more often.
3. Vacation more often. This goes along with having a healthy work/life balance.I really should utilize all my given vacation time and use it to travel around the country and/or the world. I love travelling, but I'm often constrained by time and money, but short trips can be relatively inexpensive and are great at dealing with work stress.
So those are my resolutions for the year. Feel free to offer suggestions or add your own goals in the comments.
Sunday, December 29, 2013
Tuesday, October 1, 2013
Nexus vPC with Nortel SMLT
My previous blog post was about dual homing a Nortel switch or switch stack with a pair of Nexus 5k or 7k devices. This post will expand upon that configuration and connect our pair of Nexus switches with two stacks of Nortel 5020 switches running in an SMLT configuration.
Here's the topology:
As you can see, we are using VLAN 1000 for the IST connection, disabling spanning-tree for the IST ports, creating MLT ID 1 and designating it as an IST. The IST VLAN is used for SMLT exchanges and keepalives.
Since MLT/SMLT connections don't use any negoitiation mechanism to form the trunk, we need something that can detect a failure at layer 2 to layer 3. This is where VLACP comes in. VLACP is a Nortel proprietary protocol similar to LACP and used in conjunction with MLT and SMLT connections. Like LACP, if an interface fails to see hello packets from it's neighbor interface, it will bring that interface to a down state to prevent packet loss.
At this point, our IST should be established between the two switches. We need to configure the uplinks to the Nexus switches and assign VLANs to the trunks. We will be utilizing LACP for this connection.
As you can see, this configuration is similar to the LACP configuration from a single Nortel switch with one noticeable difference. By default, LACP creates a system id based on the built-in MAC address of a switch. Since we are connecting two switch stacks to the Nexus switches, we need to define a shared system id to use. As shown above, we start by associating the lacp key with an MLT and SMLT id. We then assign an SMLT system id. You can just use the built-in address from one of the switches.
We already covered the configuration for the Nexus port-channels in the previous blog post. The configuration is identical except that we are bundling more interfaces.
At this point, everything should be connected and working properly. You can verify the LACP system ID on the Nexus switches by issuing the "show lacp neighbors" command. You can also verify that the port channels are live and passed the vPC consistency check with a "show vpc".
Here's the topology:
As you can see, we have two stacks of three Nortel switches used for aggregation and server access switches. This could alternatively be a pair of Passport 8600 switches. There are three interfaces configured as the inter-switch trunk (IST). The IST is similar to the vPC peer link. Each switch on the top stack is connected to Nexus01 and each switch on the bottom stack are connected to Nexus02.
Here's the IST configuration for the Nortel switches:
Nortel-Stack01 vlan name 1000 "IST_VLAN" interface vlan 1000 ip address 10.10.10.1 255.255.255.252 2 vlan ports 1/45,2/45,3/45 tagging tagAll vlan members 1000 1/45,2/45,3/45 vlan ports 1/45,2/45,3/45 pvid 1000 vlan configcontrol strict no auto-pvid spanning-tree port 1/45,2/45,3/45 learning disable mlt spanning-tree 1 stp 1 learning disable mlt 1 name "IST" enable member 1/45,2/45,3/45 interface mlt 1 ist peer-ip 10.10.10.2 ist vlan 1000 ist enable vlacp macaddress 18:0c:20:00:00:0f vlacp port 1/45,2/45,3/45 timeout short vlacp port 1/45,2/45,3/45 enable vlacp enable Nortel-Stack02 vlan create 1000 type port 1 vlan name 1000 "IST_VLAN" interface vlan 1000 ip address 10.10.10.2 255.255.255.252 2 vlan ports 1/45,2/45,3/45 tagging tagAll vlan members 1000 1/45,2/45,3/45 vlan ports 1/45,2/45,3/45 pvid 1000 vlan configcontrol strict no auto-pvid spanning-tree port 1/45,2/45,3/45 learning disable mlt spanning-tree 1 stp 1 learning disable mlt 1 name "IST" enable member 1/45,2/45,3/45 interface mlt 1 ist peer-ip 10.10.10.1 ist vlan 1000 ist enable vlacp macaddress 18:0c:20:00:00:0f vlacp port 1/45,2/45,3/45 timeout short vlacp port 1/45,2/45,3/45 enable vlacp enable
As you can see, we are using VLAN 1000 for the IST connection, disabling spanning-tree for the IST ports, creating MLT ID 1 and designating it as an IST. The IST VLAN is used for SMLT exchanges and keepalives.
Since MLT/SMLT connections don't use any negoitiation mechanism to form the trunk, we need something that can detect a failure at layer 2 to layer 3. This is where VLACP comes in. VLACP is a Nortel proprietary protocol similar to LACP and used in conjunction with MLT and SMLT connections. Like LACP, if an interface fails to see hello packets from it's neighbor interface, it will bring that interface to a down state to prevent packet loss.
At this point, our IST should be established between the two switches. We need to configure the uplinks to the Nexus switches and assign VLANs to the trunks. We will be utilizing LACP for this connection.
On Both Switches lacp key 32 mlt-id 32 smlt-id 32 lacp port-mode advance lacp smlt-sys-id 18:0c:20:00:00:0f interface FastEthernet 1/48,2/48,3/48 lacp key 32 lacp timeout-time short lacp mode active lacp aggregation enable vlan create 2,4,8 type port 1 vlan ports 1/48,2/48,3/48 tagging untagPvidOnly vlan members 1 1/45,1/48,2/45,2/48,3/45,3/48 vlan members 2 1/45,1/48,2/45,2/48,3/45,3/48 vlan members 4 1/45,1/48,2/45,2/48,3/45,3/48 vlan members 8 1/45,1/48,2/45,2/48,3/45,3/48
As you can see, this configuration is similar to the LACP configuration from a single Nortel switch with one noticeable difference. By default, LACP creates a system id based on the built-in MAC address of a switch. Since we are connecting two switch stacks to the Nexus switches, we need to define a shared system id to use. As shown above, we start by associating the lacp key with an MLT and SMLT id. We then assign an SMLT system id. You can just use the built-in address from one of the switches.
We already covered the configuration for the Nexus port-channels in the previous blog post. The configuration is identical except that we are bundling more interfaces.
At this point, everything should be connected and working properly. You can verify the LACP system ID on the Nexus switches by issuing the "show lacp neighbors" command. You can also verify that the port channels are live and passed the vPC consistency check with a "show vpc".
Sunday, September 22, 2013
Nexus vPC to Nortel Switches
I have a customer that currently has a network with a pair of Nortel Passport 8000 switches as their core and a bunch of Nortel 5510 and 5520 access switches for server and workstation connectivity. This customer decided to purchase a pair of Nexus 5596T switches to replace the Nortel 8000 switches.
In the existing environment, they are utilizing Nortel's split multi-link trunking (SMLT) for layer 2 resiliency and aggregation. I have been tasked with setup of the new Nexus switches and configuring a similar topology using Cisco vPC technology for layer 2 resiliency and aggregation.
In this example, we are going to create vPC peer-link and a single LACP port channel to a Nortel 5510 switch.
Let's start with the basic Nexus vPC configuration. I'm not going to spend a lot of time on it as it's been covered by many people.
And now we confirm that the vPC peer-link is up and working properly.
And now that the vPC connection is up between the switches, we can configure the port-channel for the Nortel switch.
*Update: The configuration "no lacp graceful-convergence" was added to the port channel interfaces. LACP graceful convergence was introduced in version 5.1(3)N1(1) and is on by default. Cisco recommends disabling this feature on port channels connected to non NX-OS devices as it can cause the interfaces to inexplicibly enter a standby state.
And now for the multi-link trunk (MLT) configuration on the Nortel switch.
Note that I left the PVID untagged for the two uplink ports to ensure compatibility with the Nexus switches since Cisco uses the concept of native VLANs on trunk ports. For this example, we are keeping it simple and using VLAN 1 for the native VLAN.
When you configure an LACP key on the Nortel switch, it assigns the unique LACP key to an MLT starting with the highest number MLT (32 in this case) and working backwards. The LACP key is local to the switch or stack and is similar to the channel-group with the Nexus switches. The MLT ID is similar to the port-channel interface on the Nexus switches.
At this point, the port-channel and MLT should be up on all the switches. Let's confirm.
As you can see, LACP is up on all switches and vPC 10 is up on both Nexus switches. At this point we can confirm connectivity.
As we can see, ping works from each of the Nexus switches and from the Nortel switch to each Nexus switch and the shared HSRP IP.
The biggest caveat with this configuration is that you cannot change VLAN assignments on the trunk while LACP is active on the Nortel switch. If you change the allowed VLANs on either side of the trunk, it brings down the LACP neighborship until you disable and re-enable LACP on both uplinks. For this reason, I recommend you set the allowed VLANs on the port-channel on the Nexus switches to prevent the trunk from going down simply by creating a VLAN on the switch.
In the existing environment, they are utilizing Nortel's split multi-link trunking (SMLT) for layer 2 resiliency and aggregation. I have been tasked with setup of the new Nexus switches and configuring a similar topology using Cisco vPC technology for layer 2 resiliency and aggregation.
In this example, we are going to create vPC peer-link and a single LACP port channel to a Nortel 5510 switch.
Let's start with the basic Nexus vPC configuration. I'm not going to spend a lot of time on it as it's been covered by many people.
Nexus-01 feature vpc feature lacp interface mgmt0 ip address 192.168.200.253/24 vpc domain 1 role priority 4096 peer-keepalive destination 192.168.200.253 peer-gateway auto-recovery interface Ethernet1/31 descriptoin vPC Peer Link switchport mode trunk spanning-tree port type network channel-group 1 mode active interface Ethernet1/32 description vPC Peer Link switchport mode trunk spanning-tree port type network channel-group 1 mode active interface port-channel1 switchport mode trunk spanning-tree port type network vpc peer-link interface vlan2 ip address 192.168.10.254/24 standby 2 ip 192.168.10.1 standby 2 priority 110 no shutdown Nexus-02 feature vpc feature lacp interface mgmt0 ip address 192.168.200.254/24 vpc domain 1 role priority 8192 peer-keepalive destination 192.168.200.254 peer-gateway auto-recovery interface Ethernet1/31 descriptoin vPC Peer Link switchport mode trunk spanning-tree port type network channel-group 1 mode active interface Ethernet1/32 description vPC Peer Link switchport mode trunk spanning-tree port type network channel-group 1 mode active interface port-channel1 switchport mode trunk spanning-tree port type network vpc peer-link interface vlan2 ip address 192.168.10.253/24 standby 2 ip 192.168.10.1 no shutdown
And now we confirm that the vPC peer-link is up and working properly.
Nexus-01# sh vpc
Legend:
(*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 1
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : primary
Number of vPCs configured : 1
Peer Gateway : Enabled
Peer gateway excluded VLANs : -
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Enabled (timeout = 240 seconds)
vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ --------------------------------------------------
1 Po1 up 1-2
Nexus-02# sh vpc
Legend:
(*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 1
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : secondary
Number of vPCs configured : 1
Peer Gateway : Enabled
Peer gateway excluded VLANs : -
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Enabled (timeout = 240 seconds)
vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ --------------------------------------------------
1 Po1 up 1-2
And now that the vPC connection is up between the switches, we can configure the port-channel for the Nortel switch.
Nexus-01
interface Ethernet1/1
description Trunk to Nortel Test
switchport trunk allowed vlan 1-2
switchport mode trunk
channel-group 10 mode active
interface port-channel10
description Trunk to Nortel Test
no lacp graceful-convergence
switchport trunk allowed vlan 1-2
switchport mode trunk
vpc 10
Nexus-02
interface Ethernet1/1
description Trunk to Nortel Test
switchport trunk allowed vlan 1-2
switchport mode trunk
channel-group 10 mode active
interface port-channel10
description Trunk to Nortel Test
no lacp graceful-convergence
switchport trunk allowed vlan 1-2
switchport mode trunk
vpc 10
*Update: The configuration "no lacp graceful-convergence" was added to the port channel interfaces. LACP graceful convergence was introduced in version 5.1(3)N1(1) and is on by default. Cisco recommends disabling this feature on port channels connected to non NX-OS devices as it can cause the interfaces to inexplicibly enter a standby state.
And now for the multi-link trunk (MLT) configuration on the Nortel switch.
ip address switch 192.168.10.10 ip address netmask 255.255.255.0 vlan mgmt 2 vlan create 2 type port 1 vlan ports 47-48 tagging unTagPvidOnly vlan members 1 1-48 vlan members 2 47-48 vlan ports 47-48 pvid 1 interface fastEthernet 47-48 lacp key 10 lacp timeout-time short lacp mode active lacp aggregation enable
Note that I left the PVID untagged for the two uplink ports to ensure compatibility with the Nexus switches since Cisco uses the concept of native VLANs on trunk ports. For this example, we are keeping it simple and using VLAN 1 for the native VLAN.
When you configure an LACP key on the Nortel switch, it assigns the unique LACP key to an MLT starting with the highest number MLT (32 in this case) and working backwards. The LACP key is local to the switch or stack and is similar to the channel-group with the Nexus switches. The MLT ID is similar to the port-channel interface on the Nexus switches.
At this point, the port-channel and MLT should be up on all the switches. Let's confirm.
Nexus-01# sh lacp neigh
Flags: S - Device is sending Slow LACPDUs F - Device is sending Fast LACPDUs
A - Device is in Active mode P - Device is in Passive mode
PO10 neighbors
Partner's information
Partner Partner Partner
Port System ID Port Number Age Flags
Eth1/1 32768,0-19-69-a6-4c-0 0x2f 378 SA
LACP Partner Partner Partner
Port Priority Oper Key Port State
32768 0x300a 0x3f
Nexus-02# sh lacp neigh
Flags: S - Device is sending Slow LACPDUs F - Device is sending Fast LACPDUs
A - Device is in Active mode P - Device is in Passive mode
Po10 neighbors
Partner's information
Partner Partner Partner
Port System ID Port Number Age Flags
Eth1/1 32768,0-19-69-a6-4c-0 0x30 692 SA
LACP Partner Partner Partner
Port Priority Oper Key Port State
32768 0x300a 0x3f
Nexus-01# sh vpc
!------Excluded for brevity---------!
vPC status
----------------------------------------------------------------------------
id Port Status Consistency Reason Active vlans
------ ----------- ------ ----------- -------------------------- -----------
10 Po10 up success success 1-2
Nexus-02# sh vpc
!------Excluded for brevity---------!
vPC status
----------------------------------------------------------------------------
id Port Status Consistency Reason Active vlans
------ ----------- ------ ----------- -------------------------- -----------
10 Po10 up success success 1-2
5510-48T-PWR# sh lacp port 47-48
Admin Oper Trunk Partner
Port Priority Lacp A/I Timeout Key Key AggId Id Port Status
---- -------- ------- --- ------- ----- ----- ----- ----- ------- -------
47 32768 Active A Short 10 12298 8224 32 257 Active
48 32768 Active A Short 10 12298 8224 32 16641 Active
As you can see, LACP is up on all switches and vPC 10 is up on both Nexus switches. At this point we can confirm connectivity.
Nexus-01# ping 192.168.10.10 PING 192.168.10.10 (192.168.10.10): 56 data bytes 64 bytes from 192.168.10.10: icmp_seq=0 ttl=63 time=1.697 ms 64 bytes from 192.168.10.10: icmp_seq=1 ttl=63 time=1.305 ms 64 bytes from 192.168.10.10: icmp_seq=2 ttl=63 time=1.287 ms 64 bytes from 192.168.10.10: icmp_seq=3 ttl=63 time=1.329 ms 64 bytes from 192.168.10.10: icmp_seq=4 ttl=63 time=1.393 ms Nexus-02# ping 192.168.10.10 PING 192.168.10.10 (192.168.10.10): 56 data bytes 64 bytes from 192.168.10.10: icmp_seq=0 ttl=62 time=9.314 ms 64 bytes from 192.168.10.10: icmp_seq=1 ttl=62 time=3.379 ms 64 bytes from 192.168.10.10: icmp_seq=2 ttl=62 time=3.349 ms 64 bytes from 192.168.10.10: icmp_seq=3 ttl=62 time=3.354 ms 64 bytes from 192.168.10.10: icmp_seq=4 ttl=62 time=3.352 ms 5510-48T-PWR>ping 192.168.10.1 Host is reachable 5510-48T-PWR>ping 192.168.10.254 Host is reachable 5510-48T-PWR>ping 192.168.10.253 Host is reachable
As we can see, ping works from each of the Nexus switches and from the Nortel switch to each Nexus switch and the shared HSRP IP.
The biggest caveat with this configuration is that you cannot change VLAN assignments on the trunk while LACP is active on the Nortel switch. If you change the allowed VLANs on either side of the trunk, it brings down the LACP neighborship until you disable and re-enable LACP on both uplinks. For this reason, I recommend you set the allowed VLANs on the port-channel on the Nexus switches to prevent the trunk from going down simply by creating a VLAN on the switch.
Forcing a Cisco ASA Reboot
I manage a good number of Cisco ASA 5505 firewalls. I am currently working on one for a remote employee for a company who's network I manage. The firewall, at the time, was running 9.1(1)4. I discovered an issue with that code where the VPN process will occasionally cause the firewall to crash and become unresponsive for about 10-15 minutes.
In an effort to update this firewall, which happens to be in Pennsylvania (I'm in Iowa), I uploaded the new firewall and attempted to reload the device. I type reload, hit enter twice to confirm, and nothing happens! I tried it several times with different flags, including reload quick and reload noconfirm. Nothing worked! I got so desperate I even opened ASDM and attempted to reload there, thinking it might have different hooks into the underlying OS. That failed as well.
Desperate to get this firewall rebooted to fix the VPN bug (and this newly discovered reload bug, likely caused by the VPN bug), I came across the crashinfo command. To preface this, if you aren't familiar with ASAs, if they crash they dump a bunch of information to a text file on the flash drive called crash.txt. It contains a bunch of debug information, including the current memory contents and the process(es) that crashed.
The crashinfo command allows you to view or save the crash info and also allows you to simulate a crash, either by doing a test or a forced crash as you can see from the output below.
So to force a reload on the firewall, I was able to issue:
and the firewall immediately rebooted and came up on the new firmware. I wouldn't recommend doing this unless you absolutely have to, but it did bail me out in this situation.
asa(config)# crashinfo ? configure mode commands/options: console Control output of crashinfo to the console save Save exec mode commands/options: force Forcibly crash the system and reboot test Test crashinfo generation - will not crash the system asa(config)# crashinfo force ?exec mode commands/options: page-fault Crash by causing a page fault exception watchdog Crash by causing a watchdog timeout
So to force a reload on the firewall, I was able to issue:
crashinfo force watchdog
and the firewall immediately rebooted and came up on the new firmware. I wouldn't recommend doing this unless you absolutely have to, but it did bail me out in this situation.
Introduction
My name is Mike. I am currently a technical consultant/network engineer for an MSP and Service Provider in Central Iowa. I have been working in this industry for about 3 years and I am currently the lead network engineer working primarily with Cisco routers, switches, and firewalls. Due to the nature of the business however, I do find myself working with a vast range of vendors, including HP, Juniper, Dell (formerly Force10 and PowerConnect), Fortinet, Foundry, Nortel, and so on. I also work quite a bit with storage and virtualization.
The purpose of this blog is two-fold. The first is to keep a personal repository of different issues I have come upon and new technologies I learn and implement. The second purpose is the offer yet another location where fellow networkers coming up in the business can hopefully benefit from some of the topics I post about. I realize there are already more blogs than I can count, but I'm hoping to offer my own perspective and I will hopefully be beneficial to some people.
If anyone has any questions about any blog posts or anything else, feel free to reach out to be at blog@networknerdmike.com or on Twitter @networknerdmike.
The purpose of this blog is two-fold. The first is to keep a personal repository of different issues I have come upon and new technologies I learn and implement. The second purpose is the offer yet another location where fellow networkers coming up in the business can hopefully benefit from some of the topics I post about. I realize there are already more blogs than I can count, but I'm hoping to offer my own perspective and I will hopefully be beneficial to some people.
If anyone has any questions about any blog posts or anything else, feel free to reach out to be at blog@networknerdmike.com or on Twitter @networknerdmike.
Subscribe to:
Posts (Atom)