What is the OVERFLOW event in WatchService, when does it occur, and how should you handle it?
answer
- OVERFLOW = events were lost / buffer overflowed
- Delivered even if you didn't register for it
- Cause: burst of changes or slow consumer
- Fix: re-scan the directory and reconcile state
- Prevent: thin loop + offload work + debounce
basics
~20 sOVERFLOW means the watch service dropped some change events because they arrived faster than your program consumed them. When you see it, you can't trust your event list is complete, so you should re-scan the directory to find out the real current state.
solid answer
~40 sOVERFLOW is a special StandardWatchEventKinds value the service emits when events were lost — typically because changes arrived faster than the consumer drained them, or the OS notification buffer overflowed. It signals 'some events for this key were dropped; the event stream is no longer a reliable complete record.' You can receive OVERFLOW even if you didn't register for it. The correct response is not to treat it as a normal create/modify/delete but to reconcile: re-list (re-scan) the watched directory and compare against your last known state to recover anything you missed. Practically, you keep your event loop fast (offload heavy work to another thread) and debounce bursts to reduce the chance of overflow, and you always include an OVERFLOW branch that triggers a full reconciliation rather than crashing or ignoring it.
go deeper
Knows OVERFLOW means some events may have been missed and that the program shouldn't assume it saw everything.
Explains the cause (burst/slow consumer/buffer limits) and that the fix is to re-scan the directory, and that OVERFLOW can arrive unregistered.
Designs a thin event loop with work offloaded, implements reconciliation against a known-state snapshot on OVERFLOW, and tunes/justifies OS limits where relevant.
Treats OVERFLOW as part of an at-least-once-with-reconciliation delivery model, reasons about backpressure, idempotent processing, and when a fundamentally different ingestion design (queue, hashing) is safer than watching a hot directory.
## What OVERFLOW is `StandardWatchEventKinds.OVERFLOW` is one of the four standard event kinds (alongside ENTRY_CREATE, ENTRY_MODIFY, ENTRY_DELETE). Unlike the others, it does not describe a specific file change. It is a **signal that the service was unable to deliver some events** for the key — events have been **lost**. ## Why it happens Between the OS detecting a change and your code calling `pollEvents()`, the changes are held in a **bounded buffer** (the native notification queue, e.g. inotify's queue, and the JDK's per-key event list). If changes arrive in a burst faster than your loop consumes them, that buffer can fill. Common causes: - A huge number of files created/modified at once (e.g. unzipping thousands of files, a bulk copy). - A slow consumer — your loop does heavy work (parsing, network calls) inline between `take()` and `reset()`, so events pile up. - OS-level limits (on Linux, `inotify` queue size limits such as `max_queued_events`). When the buffer overflows, the service collapses the lost events into a single **OVERFLOW** event rather than silently lying about a complete stream. ## A subtle but important point **You can receive OVERFLOW even if you never registered for it.** The event kinds you pass to `register()` filter the *content* events, but OVERFLOW is always possible because it is a meta-signal about delivery, not a file operation. So every robust consumer must handle it regardless of what it registered for. ## How to handle it correctly The key realization: after OVERFLOW, **your in-memory picture of the directory may be wrong** — files may have appeared or disappeared without you seeing their events. The only safe recovery is **reconciliation**: 1. Detect OVERFLOW in your event loop. 2. Re-scan the watched directory (e.g. `Files.list(dir)` / `Files.newDirectoryStream(dir)`). 3. Diff that listing against your last-known state to figure out what actually changed (new files to process, deletions to record). 4. Update your state and carry on; still call `reset()` afterward. ```java for (WatchEvent<?> e : key.pollEvents()) { if (e.kind() == StandardWatchEventKinds.OVERFLOW) { reconcileDirectory(dir); // re-list + diff against known state continue; } handle(e); } key.reset(); ``` ## How to reduce overflow in the first place - **Keep the watch loop thin.** Do not do expensive work between `take()` and `reset()`. Instead, hand the file path off to a worker queue/executor and immediately loop again so events drain quickly. - **Debounce / coalesce bursts.** A single logical save can emit several MODIFY events; collecting events over a short window and processing the latest reduces pressure. - **Tune OS limits** if you genuinely watch high-churn directories (e.g. raising inotify queue/instance limits on Linux). ## Why ignoring it is dangerous If you treat OVERFLOW as 'nothing happened' (or skip it), you can permanently miss files — an importer would never process a dropped-in file, a build tool would not rebuild. Conversely, treating it as a create/modify on some phantom file is meaningless because OVERFLOW has no reliable `context()`. The only correct stance is: *I lost track; let me re-derive the truth from the filesystem.*
- Can you receive OVERFLOW if you only registered for ENTRY_CREATE?Yes. OVERFLOW is a delivery meta-signal, not a file operation, so it can arrive regardless of which content kinds you registered for; robust code always handles it.
- Why shouldn't you do heavy processing inside the watch loop?Heavy inline work slows the rate at which you drain events, letting the OS buffer fill and increasing the chance of OVERFLOW and lost events; offload the work to a separate thread/queue and keep the loop fast.
It's the filesystem saying 'I sent you more letters than your mailbox could hold' — you don't know which got dropped, so you walk over and check the whole mailbox yourself.
saying these in an interview costs you the question
- Ignoring OVERFLOW or treating it as a no-op (silently losing files)
- Trying to read context() of an OVERFLOW event as a real file
- Assuming OVERFLOW can't happen because you didn't register for it
- Recovering by 'just continuing' without re-scanning the directory
- Believing more inotify tuning alone removes the need to handle OVERFLOW