10.2.7.2. How to Check When a Failure Occurs

Operation Confirmed Version:
 Brocade 5600vRouter Version4.2R1S1
This section describes how to check logs and commands when a failure occurs on Brocade 5600vRouter.

VRRP failure

VRRP switching occurs when the node is restarted or when keep-alive packets (etc.) between VRRP-operational firewalls are stopped.
  • Log with the VRRP not enabled (not set)

user-admin@FW01:~$ show vrrp
VRRP isn't running
  • Log with the VRRP not enabled (not set)

user-admin@FW01:~$ show vrrp detail
--------------------------------------------------
user-admin@FW01:~$
user-admin@FW01:~$
  • Display when state transition from Backup to Master has occurred. (It can be known that the transition occurred 27 seconds ago.)

user-admin@FW02:~$ show vrrp detail
--------------------------------------------------
Interface: dp0s4
--------------
  Group: 10
  ----------
  State:                        MASTER
  Last transition:              27s

  Version:                      2
  RFC Compliant
  Virtual MAC interface:        dp0vrrp1
  Address Owner:                no

  Source Address:               172.16.1.32
  Configured Priority:          150
  Effective Priority:           150
  Advertisement interval:       20 sec
  Authentication type:          none
  Preempt:                      enabled

  VIP count:                    1
    172.16.1.33/32

user-admin@FW02:~$
  • Syslog confirmation :~$ show log vrrp

Use to check VRRP-related syslogs.
The syslog below is one which was output when the state of the secondary machine shifted from Backup to Master in terms of the VRRP.

Note

  • The syslog level specified here is Info.

  • An Info-level syslog outputs and complies a lot of information, thus be careful when using it.

2017-03-27T16:02:49.307939+00:00 FW03 Keepalived_vrrp[3359]: VRRP_Instance(vyatta-dp0s4-10) Transition to MASTER STATE
2017-03-27T16:02:49.308276+00:00 FW03 Keepalived_vrrp[3359]: VRRP_Instance(vyatta-dp0s4-10) Entering MASTER STATE
2017-03-27T16:02:49.308452+00:00 FW03 Keepalived_vrrp[3359]: VRRP_Instance(vyatta-dp0s4-10) setting protocol VIPs.
2017-03-27T16:02:49.308652+00:00 FW03 Keepalived_vrrp[3359]: VRRP_Instance(vyatta-dp0s4-10) Sending gratuitous ARPs on dp0vrrp1 for 172.16.1.33
2017-03-27T16:02:49.308821+00:00 FW03 Keepalived_vrrp[3359]: VRRP_Instance(vyatta-dp0s4-10): Sending SNMP notification
2017-03-27T16:02:49.308992+00:00 FW03 Keepalived_vrrp[3359]: VRRP_Instance(vyatta-dp0s4-10): Sending SNMP notification vrrpTrapNewMaster
2017-03-27T16:02:54.308316+00:00 FW03 Keepalived_vrrp[3359]: VRRP_Instance(vyatta-dp0s4-10) Sending gratuitous ARPs on dp0vrrp1 for 172.16.1.33
The syslog below is one which was output when the state of the secondary machine shifted to Backup through the preempt function, following recovery of the primary machine, which had been in the Master state.
2017-03-27T16:04:01.230464+00:00 FW03 Keepalived_vrrp[3359]: VRRP_Instance(vyatta-dp0s4-10) Received higher prio advert
2017-03-27T16:04:01.231042+00:00 FW03 Keepalived_vrrp[3359]: VRRP_Instance(vyatta-dp0s4-10) Entering BACKUP STATE
2017-03-27T16:04:01.231244+00:00 FW03 Keepalived_vrrp[3359]: VRRP_Instance(vyatta-dp0s4-10) removing protocol VIPs.
2017-03-27T16:04:01.271228+00:00 FW03 Keepalived_vrrp[3359]: VRRP_Instance(vyatta-dp0s4-10): Sending SNMP notification
2017-03-27T16:04:01.271418+00:00 FW03 Keepalived_vrrp[3359]: Netlink reflector reports IP fe80::200:5eff:fe00:10a removed

IPsec failure

IPsec Site-to-Site setting

  • Log which is output when the vti interface fails.

user-admin@FW03:~$ show vpn ipsec sa
Peer ID / IP                            Local ID / IP
------------                            -------------
153.xxx.xx.182                          153.xxx.xxx.227

    Tunnel  State  Bytes Out/In     Encrypt       Hash    A-Time  L-Time  Proto
    ------  -----  -------------  ------------  --------  ------  ------  -----
    vti     down   0.0/0.0        n/a           n/a       0       n/a     all

Peer ID / IP                            Local ID / IP
------------                            -------------
153.xxx.xx.226                          153.xxx.xxx.227

    Tunnel  State  Bytes Out/In     Encrypt       Hash    A-Time  L-Time  Proto
    ------  -----  -------------  ------------  --------  ------  ------  -----
    vti     up     0.0/0.0        aes256        sha1      263     3600    all
  • The IPsec vti interface is indicated as AdminDown.

user-admin@FW03:~$ show interfaces

Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface        IP Address                        S/L  Description
---------        ----------                        ---  -----------
dp0s3            100.xx.xx.60/20                   u/u
dp0s4            -                                 A/D
dp0s5            -                                 A/D
dp0s6            153.xxx.xxx.227/29                u/u
dp0s7            192.168.1.12/28                   u/u
vti0             10.1.1.2/30                       A/D
vti1             10.2.1.2/30                       u/u
user-admin@FW03:~$
  • SNMP Trap confirmation

An SNMP Trap is sent when the IPsec-connected opposite firewall fails.
OID:1.3.6.1.6.3.1.1.5.4 -> LinkDown
OID:1.3.6.1.2.1.2.2.1.2 -> ifIndex
The SNMP Trap below allows to know LinkDown regarding “ifIndex=14”.
fwope_ipsec_vtidown_fig1
The ifIndex value can be identified with the following command. Since the value of the vti0 interface is 14, it can be known through the SNMP Trap that the vti0 interface downed.
user-admin@FW30:~$ show interfaces system enabled
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 promiscuity 0
    RX: bytes  packets  errors  dropped overrun mcast
    354160     5394     0       0       0       0
    TX: bytes  packets  errors  dropped carrier collsns
    354160     5394     0       0       0       0
    link/ether fa:16:3e:a1:7c:9e brd ff:ff:ff:ff:ff:ff promiscuity 0
    tun
    RX: bytes  packets  errors  dropped overrun mcast
    218221     2241     0       0       0       0
    TX: bytes  packets  errors  dropped carrier collsns
    215034     1250     0       0       0       0
9: dp0s4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DORMANT group default qlen 1000
    link/ether fa:16:3e:48:38:4d brd ff:ff:ff:ff:ff:ff promiscuity 0
    tun
    RX: bytes  packets  errors  dropped overrun mcast
    85324      896      0       0       0       0
    TX: bytes  packets  errors  dropped carrier collsns
    37992      334      0       0       0       0
12: dp0s7: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DORMANT group default qlen 1000
    link/ether fa:16:3e:88:4b:d5 brd ff:ff:ff:ff:ff:ff promiscuity 0
    tun
    RX: bytes  packets  errors  dropped overrun mcast
    376482     3693     0       0       0       0
    TX: bytes  packets  errors  dropped carrier collsns
    371776     2402     0       0       0       0
14: vti0@NONE: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1428 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1
    link/ipip 172.16.110.30 peer 172.16.210.40 promiscuity 0
    vti remote 172.16.210.40 local 172.16.110.30 ikey 144.0.0.1 okey 144.0.0.1
    RX: bytes  packets  errors  dropped overrun mcast
    1176       12       0       0       0       0
    TX: bytes  packets  errors  dropped carrier collsns
user-admin@FW30:~$

Instance restart

  • Syslog confirmation :~$ show log

The syslog message below is output when restart is executed through the Customer Portal.
2017-03-27T16:01:41.396055+00:00 FW01 systemd-logind[1661]: Power key pressed.
2017-03-27T16:01:41.396562+00:00 FW01 systemd-logind[1661]: Powering Off...
2017-03-27T16:01:41.396762+00:00 FW01 systemd-logind[1661]: System is powering down.
  • SNMP Trap confirmation

The OID SNMP Trap below is sent when firewall shutdown occurs through restart.

OID:1.3.6.1.4.1.8072.4.0.3 -> nsNotifyShutdown

fwope_shutdown_Fig1

Note

To make the SNMP Trap above be sent, SNMP settings are needed.