Check Point and F5© BIG-IP© LTM© Alert of the Week: RX traffic drastically reduced post fail over, possible ARP issue
Notice: This blog post was originally published on Indeni before its acquisition by BlueCat.
The content reflects the expertise and perspectives of the Indeni team at the time of writing. While some references may be outdated, the insights remain valuable. For the latest updates and solutions, explore the rest of our blog
Key takeawaysThis key takeaway was generated through LLMs crawling the page and coming up with an overview of the content.The article describes an indeni alert for Check Point ClusterXL (with F5 BIG-IP LTM also referenced) that detects potential Layer 2 failover awareness issues when a cluster member becomes active. It explains that after a failover the newly active device received far fewer packets than the previous active member, indicating surrounding network equipment may not have updated MAC-to-VIP mappings via gratuitous ARP. The operational impact is potential traffic blackholing after failover and the article points to manual remediation steps and Check Point SK50840 for further guidance.
What specific symptom triggers the indeni failover alert in a ClusterXL environment?
Indeni triggers the failover alert when it observes that, immediately after a ClusterXL failover, the newly active cluster member receives a substantially lower number of packets compared to the pre-failover active member over a similar time window (the example shows 0 packets versus 104,462 packets). This disparity indicates the new active device is not seeing remotely similar traffic levels, suggesting the surrounding network equipment may not be aware of the failover at Layer 2.
Why might traffic stop reaching the newly active cluster member after a failover?
Traffic may stop reaching the new active member because virtual IP responsibility shifts and MAC addresses change during failover. ClusterXL issues gratuitous ARP messages to inform upstream devices of the MAC change, but some network equipment can fail to process those ARPs correctly. If upstream switches or routers do not update their MAC-to-VIP mappings, they will continue sending traffic to the previous MAC, causing the new active member to receive little or no traffic.
What manual remediation steps or references does the article recommend when this alert occurs?
The article recommends reviewing Check Point document SK50840 for further information about ClusterXL gratuitous ARP behavior and interoperability with network equipment. It also describes the general approach: verify that gratuitous ARPs are issued during failover and confirm upstream devices update MAC-to-VIP mappings; if they do not, adjust network equipment behavior or configuration so it accepts or learns the new MAC. The article additionally points readers to indeni’s guide to Preemptive Maintenance of Check Point Firewalls for more context.
NOTE: The alert detailed below is given with a Check Point ClusterXL example, although F5 BIG-IP LTM is covered for this issue as well (see SOL7332).
This is a real life sample alert from indeni
Description:
A fail over was identified at Device time: Jul 18 03:02 2014 UTC, indeni time: Jul 18 03:02 2014 UTC. This device is now the active member of the cluster and in the period immediately following the fail over (3 minutes more or less) it received 0 packets compared to 104462 packets that were received by jcnj-fw2 (10.10.10.2) in a similar amount of time immediately BEFORE the fail over. This indicates the possibility that the surrounding network equipment may not be aware of the fail over on the layer 2 level.
Manual Remediation Steps:
It is possible this is caused by the fact that during a fail over the responsibility for the virtual IPs moves from one cluster member to the other and the MAC addresses change. ClusterXL issues gratuitous arps to deal with this but it may not work with your equipment. Please review SK50840 for more information.
How does this alert work?
indeni monitors the traffic passing through all members of an HA cluster. If it sees that post a failover the newly active member isn’t seeing remotely similar levels of traffic as the pre-failover active member did, the alert is triggered.
Interested in learning more? Download for free the official indeni guide to Preemptive Maintenance of Check Point Firewalls. Just fill out the form below:
[ninja_form id=5]
Related content
BlueCat DDI data boosts Cisco Cloud Control AI-driven operations
BlueCat’s integration with Cisco Cloud Control provides AI agents with access to trusted DDI data for network investigation and remediation.
Automate your DDI modernization path by migrating with Micetro
Automate cross-platform DNS and DHCP migration with Micetro to reduce risk, eliminate manual effort, and modernize infrastructure faster.
How to map your network with user-defined links in Integrity X
Map your network with user-defined links in Integrity X to define and manage custom relationships, such as dual-stack and NAT environments.
Your journey to intelligent NetOps begins at Cisco Live
Visit BlueCat’s booth or book a meeting now to learn more about how our solutions can help you build a network that supports constant change.