Why does $unwind after $lookup drop documents, and how do you keep them?

## The idiom and its trap Because `$lookup` always produces an array, a pipeline that joins a one-to-one relationship usually wants to flatten it: ```js { $lookup: { from: "customers", localField: "customerId", foreignField: "_id", as: "customer" } }, { $unwind: "$customer" } ``` After this, `customer` is a sub-document rather than a one-element array, and `"$customer.name"` works. That is the reason the pair is so common. The trap is what it does to unmatched documents. ## Why documents vanish `$unwind` deconstructs an array field, emitting **one output document per element**. Three input shapes produce zero output documents in its default mode: - the field holds an **empty array** — no elements, so nothing to emit, - the field is **missing**, - the field is **null**. A `$lookup` that matched nothing sets `as` to `[]`, which is exactly the first case. So an order whose customer record was deleted quietly disappears from the report. The left-outer semantics that `$lookup` carefully preserved are thrown away by the very next stage — and no error is raised, so the only symptom is a row count that is lower than expected. ## Restoring outer semantics ```js { $unwind: { path: "$customer", preserveNullAndEmptyArrays: true } } ``` With this option, documents whose path is missing, null, or an empty array are passed through. For the empty-array case, the output document simply **does not contain the field** — it is not set to `null` or `[]`. That matters downstream: a later `$match: { "customer.tier": "gold" }` will exclude those documents anyway, while `$ifNull` or `$exists` lets you handle them deliberately. If you specifically want a null placeholder, add `{ $set: { customer: { $ifNull: [ "$customer", null ] } } }` after the unwind. If all you need is to flatten a genuinely one-to-one join without any risk of dropping documents, `{ $set: { customer: { $first: "$customer" } } }` is a smaller hammer: it yields the sub-document when there was a match and `null` when there was not, with no fan-out semantics involved at all. ## Fan-out on the many side When the join is one-to-many, `$unwind` does not just flatten — it multiplies. One order with four matching line items becomes four output documents, each carrying a full copy of the order's own fields. That is intended when you want a flat, row-per-pair shape for grouping or export, but it is a real cost: the order's fields are duplicated per element, and any later `$group` has to reassemble what you just took apart. If you only need a count or a sum over the joined side, computing it inside the `$lookup` sub-pipeline is cheaper than unwinding and regrouping. ## The coalescence optimization MongoDB's pipeline optimizer recognizes the specific pattern of an `$unwind` that immediately follows a `$lookup` and operates on that lookup's `as` field. When it does, it can **coalesce** the two stages into one, so the joined documents are streamed out one at a time instead of being assembled into a full array first. This matters when a single input document matches a very large number of foreign documents: without coalescence the intermediate array has to exist as a value inside one document and is bound by the 16 MB BSON document limit; with it, that array is never built. Two practical consequences follow. First, keep the `$unwind` **immediately** after the `$lookup` — inserting a `$project` or `$addFields` between them can prevent the coalescence. Second, do not read too much into stage counts when you inspect a plan: the two stages appearing as one is the optimization working, not a missing stage. ## Ordering guidance As a rule: `$match` on the left side goes *before* the `$lookup`; a `$match` on joined fields goes *after* the `$unwind` (or, better, inside the lookup's sub-pipeline so the foreign side is filtered at the source rather than after being fetched and flattened).