After an IPv4 failover the new holder sent gratuitous ARPs, yet some hosts on the same LAN keep sending to the old MAC for minutes; what in ARP explains it?
answer
- the receiver decides, not the sender
- hardening against unsolicited ARP
- reply form, timing, a lost broadcast
- a holder that never died
- falls back to the cache's own ageing
basics
~20 sGratuitous ARP only works if each receiver accepts it. Hosts hardened to ignore unsolicited ARP, announcements lost or sent too early, or an old holder still answering leave caches stale until each host's own entry ages out.
solid answer
~50 sA gratuitous ARP is advice, not a command: RFC 826 says a receiver merges it, but nothing confirms that it did. Minutes of staleness usually mean the hosts fell back to their own cache ageing, for one of a few reasons. **Receiver policy**: some implementations, as poisoning hardening, ignore unsolicited ARP or lock an entry for a while after an update. **Packet form**: a reply-form or unicast announcement is dropped by implementations that expect replies only to their own requests, as RFC 5227 notes. **Delivery**: it was sent once, or before the standby's port forwarded, and broadcast has no acknowledgement. **A live old holder**: the old node still answers ARP for the address, so caches flip back. Fixes follow the cause: repeat request-form announcements after the link is up, or move a shared virtual MAC so host caches never need changing.
go deeper
Remember that an ARP announcement is unacknowledged broadcast and that each host chooses whether to accept it, so stale hosts after a failover are expected, not mysterious.
Explain the main causes: receivers that ignore unsolicited ARP, reply-form announcements some implementations drop, lost or early broadcasts, and that ignored entries wait for the cache's own ageing.
Diagnose from the wire: was the announcement on the host's segment, was it well formed, is another node still claiming the address, and is the stale party a router serving remote clients.
Choose a design that does not depend on receivers' ARP policy, such as moving a virtual MAC, and treat relaxing unsolicited-ARP hardening as a security trade with an owner.
## Why "we sent it" proves little A **gratuitous ARP** (an ARP Announcement: a broadcast ARP packet whose sender IP and target IP are both the announced IPv4 address) is fire-and-forget. RFC 826 tells a receiver to overwrite an existing cache entry with the sender's MAC, but there is no acknowledgement, no retry built into the protocol, and no way for the sender to learn which hosts complied. A host that ignores the announcement keeps its old entry until its own ageing removes it. RFC 1122 requires every ARP implementation to have such a flush mechanism, and says a timeout SHOULD be configurable, but fixes no value; lifetimes of a few minutes are a common implementation choice, which matches the symptom. ## The usual causes | Cause | What happens | How it shows up | |---|---|---| | **Receiver ignores unsolicited ARP** | hardening against cache poisoning: accept a mapping only from a reply to a request it sent, refuse to change an entry updated very recently, or refuse ARP-driven updates entirely | the announcement is on the wire at the host, but its cache still holds the old MAC | | **Announcement in reply form** | RFC 5227 lists incorrect implementations that drop replies they did not solicit, replies arriving by broadcast, or replies whose target fields do not match them | hosts of one kind update and hosts of another do not | | **Lost or early broadcast** | sent once, or before the standby's link and switch port were forwarding; broadcast is unacknowledged | no announcement ever reached the host's segment | | **Old holder still alive** | the "failed" node keeps answering ARP Requests or sending its own ARP for the address | caches flip back and forth; a node implementing RFC 5227 also detects an address conflict | | **Host is not the stale party** | the client sits on another subnet and reaches the address through a router | the router's ARP entry is stale, not the client's | | **Mapping not learned from the wire** | in some virtualised networks a controller supplies ARP answers and drops announcements (an implementation choice) | no host or router changes, whatever is sent | ## Reasoning from the protocol 1. **Was the announcement present on the affected host's segment?** Capture there. If it never arrived, the problem is delivery or timing, not the host. 2. **What did it look like?** Check that it was an ARP Request (opcode `1`) to `ff:ff:ff:ff:ff:ff`, with sender IP = target IP = the moved address and sender hardware address = the new MAC. A reply-form, unicast or malformed announcement explains selective failure. 3. **Did the host already have an entry?** Under RFC 826 an announcement updates an existing entry; a host without one simply resolves the address when needed and gets the new holder's answer. 4. **Does anyone else still claim the address?** ARP traffic from the old MAC with the moved IP as sender means two holders, and no announcement can win that. 5. **Is the host's policy to ignore it?** If the packet arrived intact and the cache did not change, the receiver chose not to merge it, and only its ageing or an explicit flush will move it. ## What actually fixes it - **Send it well**: request form, broadcast, repeated a few times (RFC 5944 says SHOULD retransmit gratuitous ARPs a small number of times), and repeated again once the link is known to be forwarding. - **Make host caches irrelevant**: move a **shared virtual MAC** with the address, so host entries stay correct and only switch MAC tables must relearn, which they do from the frame's source address whatever the hosts' ARP policy. - **Guarantee a single holder**: fencing the old node is the redundancy mechanism's job; ARP's last-writer-wins merge cannot arbitrate. - **Accept the trade knowingly**: the hardening that ignores unsolicited ARP is a real defence against poisoning, and relaxing it on a segment that relies on announcement-based failover is a security decision, not a tuning tweak. ## The point an interviewer is after The sender controls only what goes on the wire; each receiver's ARP policy decides what happens to it. A failover design that needs every host to accept unsolicited ARP is betting on settings it does not own, and the strongest designs avoid needing host caches to change at all.
- Why does a shared virtual MAC sidestep hosts that ignore unsolicited ARP?Because their caches never become wrong. The virtual IP keeps mapping to the same virtual MAC, so a host that ignores every announcement is still correct. What must change is the switches' MAC tables, and switches learn a MAC's port from the source address of any frame, including the announcement, regardless of any host's ARP policy.
- Why is ignoring unsolicited ARP a reasonable default on some hosts despite this failure?RFC 826's merge rule believes any sender, so an unsolicited packet claiming the gateway's address can redirect a host's traffic. Refusing unsolicited updates closes that cheap path at the cost of slower recovery when an address genuinely moves; the host then relearns through its own request once the entry ages out.
saying these in an interview costs you the question
- If the gratuitous ARP was sent, every host on the LAN has updated.
- Hosts ignoring the announcement must have a broken ARP implementation.
- Sending the announcement as an ARP Reply makes more hosts accept it.
- Static ARP entries on clients make virtual-IP failover faster.
- A remote client's own ARP cache is what goes stale after the move.