Job postings expire hourly but the posting-vector index rebuilds nightly — how does the retrieval stage stay fresh?

## What actually goes stale "The index is stale" hides three separate clocks. Keep them apart: - **Index freshness** — whether the postings currently open are the postings in the index. This is what a nightly build costs you. - **Vector staleness within a posting** — a posting whose text was edited after it was embedded. Usually tolerable, and repaired by re-embedding on the edit event. - **Encoder version** — what the vectors mean at all. A different clock entirely, and a much sharper failure. This leaf's operational problem is the first one, in both directions: postings arriving and postings leaving. ## Additions: a fresh segment beside the base A nightly build gives a posting an invisible window of up to twenty-four hours, which on a job board is the window where it gets most of its applications. The standard shape is two segments searched together: - the **base index**, rebuilt on a schedule over the whole catalogue, compacted and well-formed; - a **fresh segment**, small, receiving upserts from an event-driven embedding path as postings are ingested, and searched on every request beside the base. The retrieval stage queries both and merges by similarity. Because the fresh segment holds hours rather than years of inventory, it stays small enough that searching it adds little latency; at the next rebuild its contents are folded into the base and it starts empty again. The price is a second search per request and a merge, which is a fixed and modest cost. ## Removals: enforce on the read path Removal is the asymmetric case. A seeker who applies to a role that was filled last week has a worse experience than a seeker who never saw it, and the damage lands on the employer too. But most index structures make deletion lazy — a delete marks an entry and the structure reclaims it at compaction — and structures differ in exactly how. The tier therefore cannot wait for the index to forget: 1. The posting lifecycle publishes a close, withdraw or expiry event. 2. That id enters a **tombstone set** the retrieval stage can check in constant time. 3. Every candidate coming out of either segment is checked against the tombstone set and its own expiry time, and dropped before it reaches the scorer. 4. The next rebuild removes the vector physically and the tombstone entry retires with it. The rule to state plainly: **deletion is enforced on the read path; the index catches up later.** ## Tombstones cost depth, not correctness A dead entry still occupies a slot in the search's result set, so as tombstones accumulate the stage returns fewer live candidates than the depth it asked for. If a fifth of every result set is dead, a search at depth 900 yields roughly 720 live candidates before eligibility filters take their share — which is a recall loss no error will announce. Two responses: - raise depth to compensate, which costs latency and merge work; - treat the **tombstone ratio** as a build trigger: past a threshold, rebuild off-cycle rather than keep paying. ## What to monitor - **Invisible window**, at p95: elapsed time from posting created to first retrievable. This is the number the business cares about. - **Tombstone ratio** in returned result sets, and time since the last full rebuild. - **Effective depth**: live candidates surviving the tombstone check, against the depth requested. - **Fresh-segment size and lag**: a stalled embedding path shows up as a segment that stops growing, which looks like nothing at all on request-level metrics. ## The trade being made A more frequent full rebuild is the simplest answer and is sometimes right: if the catalogue is small enough that a rebuild takes minutes, the two-segment design is unnecessary complexity. The segment split earns its keep when the rebuild is expensive relative to the freshness the product needs — which on a multi-million-posting board with hourly turnover it almost always is.