How does a BSON reader skip a field it does not want, and what must it still walk?
answer
- hop, do not parse
- the type byte decides the jump
- the name still gets scanned
- no index, no sorted keys
basics
~20 sA BSON reader reads an element's type byte, scans its name to the NUL, then jumps the value using a fixed width or its int32 length. With no field index, it still visits every earlier sibling.
solid answer
~40 sA BSON reader moves one element at a time. It reads the type byte, scans the `e_name` to its NUL, then computes the value's width: fixed for types like `int32` or boolean, from an `int32` for strings and nested documents. A nested document is the payoff, since one four-byte read jumps past it, whatever it holds. What the reader cannot avoid is the walk itself: BSON has no field index and keeps pairs in the writer's order rather than sorted, so reaching a field means hopping over every sibling before it, and a missing field means hopping to the terminator. Array element 500 costs 500 hops. Skipped bytes also go unchecked, so a jump proves nothing about what it passed.
code
plaintext · 3 linesnext = value_start + fixed_width(type) int32, boolean, double, null...
next = value_start + 4 + int32_at(value_start) string
next = value_start + int32_at(value_start) embedded document, arraygo deeper
Remember that a reader can pass over a nested document in one jump because the document starts with its own length, and that it finds fields by walking rather than by looking them up.
Walk through one hop: type byte, name scan to NUL, width from the type, jump. Say which lengths include themselves. Then name what is still walked: earlier siblings, every key byte, all elements of an array.
Say what skipping does not promise. Jumped bytes are not validated, a wrong length misaligns the reader, and an unrecognised type code stops traversal because an element has no generic length.
The tradeoff is that traversal is cheap past large nested values and linear in the number of fields, so document shape affects read cost. That is worth saying when a team debates very wide documents.
## What skipping means In BSON, **skipping** means moving past a value without decoding it. It does not mean jumping straight to a field. The specification's FAQ gives the reason the format carries extra length information at all: lengths of strings and subobjects make traversal faster, and that traversal is what a reader spends them on. ## One hop, step by step From the start of an element, a reader does the following: 1. **Reads the type byte.** A `0x00` here is the terminator, and the reader is done with this document. 2. **Scans the `e_name`.** The key has no length field, so the reader moves byte by byte to the first NUL, comparing against the key it wants if it is searching. 3. **Works out the value's width** from the type byte, using the table below. 4. **Jumps**, and is now at the next element's type byte. ## How far to jump | Value | Bytes to jump | Why | |---|---|---| | `int32`, boolean, `double`, ObjectId, `decimal128` | 4, 1, 8, 12, 16 | Fixed by the type | | null, undefined, min key, max key | 0 | No value bytes | | string, JavaScript code, symbol | 4 + `int32` | The `int32` counts text and NUL, not itself | | embedded document, array | `int32` | The `int32` counts itself and everything inside | | binary | 4 + 1 + `int32` | The `int32` counts the payload only | | regular expression | scan two `cstring`s | No length is stored | The trap is in the rows that carry an `int32`: **the same-looking prefix means different things**. From the start of a nested document, you add the `int32`. From the start of a string, you add the `int32` plus four. Mixing them up misaligns everything that follows. ## A worked trace Take the 24-byte `{"a": 1, "b": [true]}` and search for a key `"c"` that is not there: ```plaintext 0x04 type 0x10 name "a" ends at 0x06 int32, jump 4 next at 0x0B 0x0B type 0x04 name "b" ends at 0x0D read 9 at 0x0E next at 0x17 0x17 type 0x00 terminator, position now 0x18 = declared length 24 ``` The jump at `0x0E` is 14 + 9 = 23, or `0x17`. It adds no extra four, because the array's 9 includes its own prefix. The bytes from `0x12` to `0x16`, the inside of the array, were never read. ## What a reader must still walk The length prefix gives a lot, but not everything: - **Every earlier sibling.** There is no offset table, so a field at position `k` costs `k` hops at its level. - **Every sibling, for a missing field.** BSON pairs are ordered but not sorted, so nothing lets a reader stop before the terminator or binary-search. - **Every key byte along the way.** Long keys slow every hop, since each is scanned to its NUL. - **Each level of a path.** Reaching `x.y.z` is a walk at the root, then one in `x`, then one in `y`. The siblings at each level are hopped, not entered. - **Array elements too.** Keys are numeric, but the format offers no shortcut: reading element 500 means hopping 500 elements, because elements can differ in width and the key's width changes at 10 and 100. So a large subdocument costs a skipping reader one read, and a document with many small top-level fields costs a hop per field. The first is the case the layout makes cheap. ## What skipping does not promise - **It validates nothing it passes over.** A skipped subdocument could hold garbage, and a reader that jumps past it finds out only if something later reads inside it. - **It needs every type's size rule.** An element has no generic length, so a type byte the reader does not recognise leaves it no way to find the next element. The corpus test vectors list an invalid type code as a decode error. - **It trusts a declared number.** A wrong length sends the reader to the wrong byte, and that byte is then read as a type code. ## Why this trade was made The layout optimises a reader that wants a few fields from a big document. It is forward traversal, not random access, and it is worth stating the distinction exactly: getting a document's declared size is a single four-byte read, and getting a named field is a walk.
- Why can a BSON reader not simply step over an element whose type byte it does not recognise?An element has no generic length field. Its width is known only through the rule for its type, so an unknown code leaves the reader with no way to find the next element, and traversal of that document has to stop. The corpus lists invalid type codes as decode errors.
- If a BSON array holds only int32 values, can a reader compute the offset of element 500?The format does not offer this. Nothing declares that the array is uniform, so a reader would first have to walk it to know that. With key widths included the arithmetic is possible only once uniformity is known, so in general a reader hops the 500 elements.
A corridor of doors with no directory at the entrance. Each door tells you how far it is to the next, so you pass rooms without going in, but you still walk the corridor door by door to find the one you want.
- The length prefix lets a reader jump straight to any named field.
- BSON keeps an index of field names at the front of each document.
- Skipping a subdocument means walking its elements to find where it ends.
- Element 500 of any BSON array can be reached directly, because keys are numeric.
- Bytes a reader jumped over have been checked as well-formed BSON.
- Keys are stored sorted, so a reader can binary-search a document.