Sunday, September 22, 2013

Forcing a Cisco ASA Reboot

I manage a good number of  Cisco ASA 5505 firewalls. I am currently working on one for a remote employee for a company who's network I manage. The firewall, at the time, was running 9.1(1)4. I discovered an issue with that code where the VPN process will occasionally cause the firewall to crash and become unresponsive for about 10-15 minutes. In an effort to update this firewall, which happens to be in Pennsylvania (I'm in Iowa), I uploaded the new firewall and attempted to reload the device. I type reload, hit enter twice to confirm, and nothing happens! I tried it several times with different flags, including reload quick and reload noconfirm. Nothing worked! I got so desperate I even opened ASDM and attempted to reload there, thinking it might have different hooks into the underlying OS. That failed as well. Desperate to get this firewall rebooted to fix the VPN bug (and this newly discovered reload bug, likely caused by the VPN bug), I came across the crashinfo command. To preface this, if you aren't familiar with ASAs, if they crash they dump a bunch of information to a text file on the flash drive called crash.txt. It contains a bunch of debug information, including the current memory contents and the process(es) that crashed. The crashinfo command allows you to view or save the crash info and also allows you to simulate a crash, either by doing a test or a forced crash as you can see from the output below.

asa(config)# crashinfo ?
configure mode commands/options:

console  Control output of crashinfo to the console
save     Save

exec mode commands/options:
force       Forcibly crash the system and reboot
test        Test crashinfo generation - will not crash the system

asa(config)# crashinfo force ?exec mode commands/options:
page-fault    Crash by causing a page fault exception
watchdog      Crash by causing a watchdog timeout

So to force a reload on the firewall, I was able to issue:
crashinfo force watchdog

and the firewall immediately rebooted and came up on the new firmware. I wouldn't recommend doing this unless you absolutely have to, but it did bail me out in this situation.