Check Point and F5© BIG-IP© LTM© Alert of the Week: RX traffic drastically reduced post fail over, possible ARP issue
Notice: This blog post was originally published on Indeni before its acquisition by BlueCat.
The content reflects the expertise and perspectives of the Indeni team at the time of writing. While some references may be outdated, the insights remain valuable. For the latest updates and solutions, explore the rest of our blog
Key takeawaysKey takeaways are generated with AI assistance. Because automated summaries can occasionally contain errors or miss important context, always refer to the full blog post for complete information.The article describes an indeni alert for Check Point ClusterXL (with F5 BIG-IP LTM also referenced) that detects potential Layer 2 failover awareness issues when a cluster member becomes active. It explains that after a failover the newly active device received far fewer packets than the previous active member, indicating surrounding network equipment may not have updated MAC-to-VIP mappings via gratuitous ARP. The operational impact is potential traffic blackholing after failover and the article points to manual remediation steps and Check Point SK50840 for further guidance.
What specific symptom triggers the indeni failover alert in a ClusterXL environment?
Indeni triggers the failover alert when it observes that, immediately after a ClusterXL failover, the newly active cluster member receives a substantially lower number of packets compared to the pre-failover active member over a similar time window (the example shows 0 packets versus 104,462 packets). This disparity indicates the new active device is not seeing remotely similar traffic levels, suggesting the surrounding network equipment may not be aware of the failover at Layer 2.
Why might traffic stop reaching the newly active cluster member after a failover?
Traffic may stop reaching the new active member because virtual IP responsibility shifts and MAC addresses change during failover. ClusterXL issues gratuitous ARP messages to inform upstream devices of the MAC change, but some network equipment can fail to process those ARPs correctly. If upstream switches or routers do not update their MAC-to-VIP mappings, they will continue sending traffic to the previous MAC, causing the new active member to receive little or no traffic.
What manual remediation steps or references does the article recommend when this alert occurs?
The article recommends reviewing Check Point document SK50840 for further information about ClusterXL gratuitous ARP behavior and interoperability with network equipment. It also describes the general approach: verify that gratuitous ARPs are issued during failover and confirm upstream devices update MAC-to-VIP mappings; if they do not, adjust network equipment behavior or configuration so it accepts or learns the new MAC. The article additionally points readers to indeni’s guide to Preemptive Maintenance of Check Point Firewalls for more context.
NOTE: The alert detailed below is given with a Check Point ClusterXL example, although F5 BIG-IP LTM is covered for this issue as well (see SOL7332).
This is a real life sample alert from indeni
Description:
A fail over was identified at Device time: Jul 18 03:02 2014 UTC, indeni time: Jul 18 03:02 2014 UTC. This device is now the active member of the cluster and in the period immediately following the fail over (3 minutes more or less) it received 0 packets compared to 104462 packets that were received by jcnj-fw2 (10.10.10.2) in a similar amount of time immediately BEFORE the fail over. This indicates the possibility that the surrounding network equipment may not be aware of the fail over on the layer 2 level.
Manual Remediation Steps:
It is possible this is caused by the fact that during a fail over the responsibility for the virtual IPs moves from one cluster member to the other and the MAC addresses change. ClusterXL issues gratuitous arps to deal with this but it may not work with your equipment. Please review SK50840 for more information.
How does this alert work?
indeni monitors the traffic passing through all members of an HA cluster. If it sees that post a failover the newly active member isn’t seeing remotely similar levels of traffic as the pre-failover active member did, the alert is triggered.
Interested in learning more? Download for free the official indeni guide to Preemptive Maintenance of Check Point Firewalls. Just fill out the form below:
[ninja_form id=5]
Related content
Get answers from the LiveNX data you already have with LiveAssist
Learn how LiveAssist, the AI-powered add-on for LiveNX, turns your network data into fast, guided answers, with value you can see in days.
LiveWire GEN3 device offerings
BlueCat LiveWire is a packet capture and network forensics platform that provides deep packet-level visibility across hybrid enterprise environments to help…
Get total control of your DDI infrastructure with Micetro’s unified API
Stop juggling between network tools and use Micetro’s RESTful API for complete control over your DDI infrastructure and cloud environment.
BlueCat MCP servers bring network intelligence to your AI assistant
Our MCP servers provide a standard integration layer that makes it easy to connect the AI tools you already use to BlueCat products.