skip to content

How does $graphLookup traverse a hierarchy, and what do connectFromField and connectToField do?

level: seniorimportance: should knowfreq 38%

answer

  1. Repeat the join, don't do it once
  2. One field says where to go next
  3. Another says what to match on arriving
  4. The output is not shaped like a tree
  5. A number tells you how far each hop went

basics

~20 s

$graphLookup starts from the value in startWith, matches it against connectToField in the from collection, then takes connectFromField from each document it finds and repeats. All documents reached are returned flattened into one array named by as, with recursion capped by maxDepth.

solid answer

~50 s

`$graphLookup` is MongoDB's recursive join. `startWith` is an expression evaluated on the input document that supplies the first search value; that value is matched against `connectToField` on documents in `from`. For each document found, the value of its `connectFromField` becomes the next round's search value, and the process repeats until nothing new matches or `maxDepth` is reached. Everything reached across all rounds is returned **flattened into a single array** under `as` — not a nested tree, and in no guaranteed order. `depthField` adds a numeric recursion depth to each returned document, which is what you use to rebuild levels client-side, and `restrictSearchWithMatch` applies an additional query filter at every step. It handles cycles: already-visited documents are not re-traversed. Two operational constraints matter — `connectToField` should be indexed, and the stage is subject to the 100 MB memory limit and ignores `allowDiskUse`.

code

javascript · 13 lines
javascript
// Walk up: every manager above employee 42
db.employees.aggregate([
  { $match: { _id: 42 } },
  { $graphLookup: {
      from: "employees",
      startWith: "$managerId",
      connectFromField: "managerId",
      connectToField: "_id",
      as: "reportingChain",
      depthField: "level",
      maxDepth: 10
  } }
])

go deeper

for a junior

Know that $graphLookup exists for recursive parent-pointer traversal, and that it fills one array field with every document it reaches.

for a middle

Be able to name the roles: startWith seeds the search, connectToField is what you match on, connectFromField supplies the next hop, and depthField records how many hops it took.

for a senior

Interviewers expect the operational picture — index the connectToField, cap with maxDepth, prune with restrictSearchWithMatch, and know that the stage cannot spill to disk and cannot target a sharded collection.

for a principal

Own the modelling call: whether the hierarchy should be traversed on read at all, or carry a materialized ancestor path maintained on write, based on read/write ratio, tree depth and how often the structure changes.

## What it is for A hierarchy stored as `parentId` pointers — org charts, category trees, comment threads, bill-of-materials, follower graphs — cannot be resolved by `$lookup`, which does exactly one hop. `$graphLookup` performs the hop repeatedly, following the chain until it runs out. ```js db.employees.aggregate([ { $match: { _id: 42 } }, { $graphLookup: { from: "employees", startWith: "$managerId", connectFromField: "managerId", connectToField: "_id", as: "reportingChain", depthField: "level", maxDepth: 10 } } ]) ``` ## The recursion, one round at a time 1. **`startWith`** is an aggregation expression evaluated against the *input* document. Here it yields the employee's `managerId`. If it evaluates to an array, every element seeds the search. 2. That value is matched against **`connectToField`** on documents in **`from`** — round 0. The matched documents are added to the result. 3. For each newly matched document, the value of its **`connectFromField`** becomes a search value for round 1, again matched against `connectToField`. 4. Repeat until a round finds nothing new, or the depth reaches **`maxDepth`**. Swapping `connectFromField` and `connectToField` reverses the direction of travel. With `startWith: "$managerId"`, `connectFromField: "managerId"`, `connectToField: "_id"` you walk *up* to ancestors. With `startWith: "$_id"`, `connectFromField: "_id"`, `connectToField: "managerId"` you walk *down* to all descendants. Getting this pair backwards is the classic mistake and typically returns an empty array or just one document. ## The output is flat, not a tree This surprises people who expect a nested structure. `as` is a **single flat array** containing every document reached at every depth, deduplicated, in no defined order. If you need a tree, add `depthField` and reconstruct it in application code, or sort the array by depth. `depthField` numbers the first round **0**, so a direct manager is depth 0, the manager's manager is depth 1, and so on. `maxDepth` uses the same numbering — `maxDepth: 0` means "do the first lookup and stop", which makes it behave like a plain `$lookup`. ## Cycles A graph with a loop — A reports to B reports to A, a category that is its own ancestor — does not hang the stage. `$graphLookup` tracks what it has already visited and does not re-traverse it, so recursion terminates. That safety property is worth stating in an interview, because it is exactly the failure mode people fear from a recursive query. ## restrictSearchWithMatch This takes a **query document** (standard find-style syntax) applied at every step of the recursion, so you can traverse only active nodes or only nodes in one region. It is a filter on which documents may be *reached* — a node that fails the filter is not returned and is also not traversed *through*, which prunes whole subtrees. Note that it takes a query, not an aggregation expression, so you cannot use `$expr`-style comparisons against the outer document there. ## Cost and constraints - **Index `connectToField`.** Every round runs a lookup against it. Without an index each round is a collection scan, and the stage runs many rounds. - **Memory.** `$graphLookup` must stay within the 100 MB stage memory limit, and if the aggregation specifies `allowDiskUse: true`, this stage ignores it. A wide traversal on a large graph therefore fails rather than spilling — which makes `maxDepth` and `restrictSearchWithMatch` correctness *and* survivability tools, not just conveniences. - **Sharding.** The `from` collection for `$graphLookup` cannot be sharded. - **Fan-out.** The result array lives inside one output document, so it is still subject to the 16 MB document limit. Traversing a highly connected graph from a hub node can breach it. ## When to reach for something else `$graphLookup` is excellent for bounded traversals: an ancestor chain a handful of levels deep, a subtree of moderate size, a two- or three-hop neighbourhood. It is not a graph engine — unbounded traversal over a densely connected graph is a poor fit for both its memory limit and its flat output. If the hierarchy is read constantly and changes rarely, storing a materialized ancestor path array on each document turns the whole traversal into a single indexed `find`, and `$graphLookup` becomes the tool you use to rebuild that path when the tree changes.

  • How would you change a $graphLookup that walks up to ancestors so it walks down to all descendants instead?
    Swap the direction of the pointer pair. Ancestors use `startWith: "$managerId"`, `connectFromField: "managerId"`, `connectToField: "_id"`. Descendants use `startWith: "$_id"`, `connectFromField: "_id"`, `connectToField: "managerId"` — you now match documents whose `managerId` points at the current node and follow their `_id` outward. The index requirement moves with it: index `managerId` for the descendant direction.
  • Does $graphLookup terminate if the hierarchy contains a cycle?
    Yes. It tracks the documents it has already visited and does not traverse them again, so a loop ends the recursion rather than running forever. Each document still appears at most once in the output array. Cycles do, however, make the depth numbers less meaningful, since a node reachable by two paths is recorded at the depth it was first reached.
  • What happens to a $graphLookup that needs more than the 100 MB stage memory limit?
    It fails. `$graphLookup` must stay within that limit and ignores `allowDiskUse: true`, so there is no spill-to-disk escape hatch. The practical response is to constrain the traversal itself — set `maxDepth`, prune with `restrictSearchWithMatch`, or start from fewer seed documents by filtering hard before the stage.
  • Can $graphLookup join with a sharded collection?
    No — the collection named by `from` must be unsharded. That is a stricter rule than plain `$lookup`, which does support a sharded foreign collection on current versions. If the hierarchy lives in a sharded collection, the usual answers are to keep a small unsharded copy of the parent-pointer graph, or to materialize an ancestor path on each document so the traversal becomes an ordinary indexed query.

saying these in an interview costs you the question

  • Expects the output to be a nested tree
  • Swaps connectFromField and connectToField
  • Assumes a cyclic graph loops forever
  • Thinks allowDiskUse rescues a large traversal
  • Believes maxDepth counts from 1

context