How is a BSON binary value laid out, and what does its subtype byte tell a reader?
answer
- length, label, then data
- the count skips the label
- old forms: 2 and 3
- applications start at 128
- new features avoid new type codes
basics
~20 sA BSON binary value is an int32 byte count, one subtype byte, then the data; the count excludes the subtype. The subtype names what the bytes mean, such as generic, UUID or vector, so new kinds of data need no new type code.
solid answer
~40 sA `binData` element has type code `\x05`, the key, then `int32 subtype (byte*)`. The `int32` counts only the data bytes, not the subtype byte and not itself. The subtype labels the contents: 0 generic, 1 function, 2 and 3 deprecated, 4 UUID, 5 MD5, 6 encrypted, 7 compressed column, 8 sensitive, 9 vector, and 128 to 255 user-defined. The label does not change how a reader gets past the value, because the length is always there, so an unfamiliar subtype can be carried through. That is why new features can arrive as subtypes. The one trap is subtype 2, whose data starts with a second `int32`.
code
plaintext · 9 lines11 00 00 00 document length: 17
05 type code: binary
78 00 key "x" as a cstring
04 00 00 00 data length: 4 (dtype + padding + 2 elements)
09 subtype: vector
03 dtype: INT8
00 padding: 0
7F 07 elements: 127, 7
00 document terminatorgo deeper
Remember the three parts in order: length, subtype, data. Know that 0 means generic and 4 means UUID, and that binary holds raw bytes with no text encoding.
Predict bytes. Show the length excludes the subtype, recount a corpus vector aloud, and explain subtype 2's extra int32 and the inner metadata of a vector.
Say who enforces what: the grammar sets the layout, drivers validate their own specifications, and the server's conformance check reports deprecated subtypes and wrong sizes as warnings.
Weigh subtype against new type code as a compatibility decision: a subtype can reach older servers and readers untouched, at the price of hiding meaning inside an opaque blob.
## The layout BSON's binary type, often written `binData`, has type code `\x05`. After the key, the grammar is `int32 subtype (byte*)`: - **`int32`**: the number of data bytes. It does not count itself and it does not count the subtype byte. - **subtype**: one unsigned byte naming the kind of data. - **data**: exactly that many bytes, which may be zero. This is a different rule from a BSON string, whose `int32` includes a trailing NUL, and from a document, whose `int32` includes itself. Here is the corpus vector for a two-byte generic value under the key `x`: ```plaintext 0F 00 00 00 document length: 15 05 type code: binary 78 00 key "x" as a cstring 02 00 00 00 data length: 2 00 subtype: generic FF FF data 00 document terminator ``` The sum is 4 + 1 + 2 + 4 + 1 + 2 + 1 = 15, which matches the first field. ## The subtypes | Subtype | Specification's name | Note | |---|---|---| | 0 | Generic | Should be the default for drivers and tools | | 1 | Function | | | 2 | Binary (old) | Deprecated in favor of 0 | | 3 | UUID (old) | Deprecated in favor of 4 | | 4 | UUID | RFC 4122 byte order | | 5 | MD5 | | | 6 | Encrypted BSON value | Client-side encryption | | 7 | Compressed BSON column | Delta and run-length encoded | | 8 | Sensitive | A label, not encryption | | 9 | Vector | Densely packed numbers of one type | | 128 to 255 | User-defined | Both ends included | Values 10 to 127 are not assigned in BSON 1.1, so they are not free for applications: the user-defined range begins at exactly 128. ## Subtypes with an inner layout The outer layout never changes, but some subtypes define structure inside the data: - **Subtype 2** starts with a second `int32` counting the bytes after it. The corpus value `06 00 00 00 02 02 00 00 00 FF FF` has an outer length of 6, which is the inner 4-byte length plus 2 bytes of data. The corpus lists a mismatch between the two as a decode error. - **Subtype 6** uses its first data byte to name a blob format. The encryption specification lists 0 for an intent-to-encrypt marking that is never persisted, and 1 and 2 for deterministic and randomized ciphertext, which carry a 16-byte key id, the original type byte, then the ciphertext. The MongoDB 9.0 source handles further values for Queryable Encryption. - **Subtype 9** opens with a `dtype` byte and a `padding` byte, then the packed numbers. `0x03` is INT8, `0x27` is FLOAT32 and `0x10` is PACKED_BIT. So `03 00 7F 07` is the INT8 vector `[127, 7]`. Padding has to be 0 except for PACKED_BIT, where it may be 0 to 7. ## Who enforces what The BSON grammar defines the layout and little else. It does not say a subtype 4 value must be 16 bytes. Other parties add the rules: 1. **Drivers** validate what their specifications demand, such as vector padding, and PyMongo 4.19 rejects a subtype 2 value whose two lengths disagree. 2. **The server**, in MongoDB 9.0, treats a document as non-conformant under `validate` with `checkBSONConformance: true` when subtype 4 or 5 is not 16 bytes, and when subtype 2 or 3 appears at all, as deprecated. `validate` returns a warning, and the log gives the reason. 3. **The server's logging**: the 9.0 manual says literal values of subtype 8 are logged as the placeholder `###`. The bytes themselves are stored as written, so subtype 8 is a redaction label and not encryption. ## Why features arrive as subtypes A reader that meets an unknown **type code** cannot find the next element. A reader that meets an unknown **subtype** still has the length, so it can skip the value or keep it as opaque bytes. The encryption specification gives this reasoning directly: a new BSON type would be a needless backwards-breaking change, because encrypted data should be storable on older servers and is a binary blob anyway. Vectors use the same mechanism. That reasoning is stated for encryption, but the effect is general: a subtype adds a meaning without adding a layout that older readers would have to know.
- What should a reader do with a subtype it has never heard of?It can still read the length and either skip the bytes or keep them as an opaque value with the subtype attached, so they round-trip unchanged. PyMongo 4.19 does this: any subtype other than 0, 3 and 4 decodes to a `Binary` object that keeps its subtype. Refusing the whole document is not needed.
- Can a BSON binary value have no data at all?Yes. The corpus has one: data length 0, subtype 0, nothing after it, inside a 13-byte document (4 + 1 + 2 + 4 + 1 + 1). The subtype byte is still present even with no data. A negative length, by contrast, is listed as a decode error.
- Does marking a value as subtype 8 protect it?It marks the value for redaction, not encryption. In MongoDB 9.0 the manual says literal subtype 8 values are logged as `###`. The bytes are stored and returned as written, so a secret still needs real encryption, and anyone able to read the document can read the value.
- The binary length field counts the subtype byte as well as the data.
- The BSON grammar itself requires a subtype 4 value to be exactly 16 bytes.
- Subtype 8 encrypts the value it holds.
- An unfamiliar binary subtype stops a reader, the same way an unknown type code does.
- Applications may use any subtype number above 9 for their own data.
- Subtype 2 is only a different label, so its data layout matches subtype 0.
Assumes MongoDB 9.0 for the server's conformance warnings and log placeholder; the binary layout itself comes from the BSON 1.1 specification.