skip to content

MongoDB Storage & Wire Protocol

How BSON travels and round-trips in practice: Extended JSON for tools, driver type mapping, UUID representation, field order and message framing. Exports that lose types are the usual follow-up.

on this pageshow

explore

questions

6

In MongoDB Extended JSON v2, how do canonical and relaxed mode differ, and which BSON types lose their type label in relaxed mode?

level: middleimportance: should knowfreq 30%

answer

  1. two modes, one conversion table
  2. numbers are the real difference
  3. what bare JSON numbers parse back to
  4. dates keep a wrapper either way
  5. which tool defaults to which mode

basics

~20 s

Canonical Extended JSON wraps every number and date in a type wrapper. Relaxed writes Int32, Int64 and finite doubles as plain JSON numbers, and 1970-9999 dates as ISO strings. Only those numbers lose their type label.

solid answer

~40 s

Extended JSON is MongoDB's text form of BSON: JSON has no ObjectId, Decimal128 or date, so `$`-prefixed type wrappers such as `$oid` and `$numberDecimal` carry them. Canonical mode favours type preservation: an Int32 is `{"$numberInt": "42"}` and a date is `{"$date": {"$numberLong": "..."}}`. Relaxed mode favours readability, and only four rows of the conversion table differ: Int32, Int64 and finite doubles become plain JSON numbers, and dates from 1970 to 9999 become `{"$date": "<ISO string>"}`. Everything else, Decimal128 included, is identical in both. The loss is in parsing: a bare integer parses to the smallest BSON integer type that holds it, so an Int64 of 42 comes back as Int32. `mongoexport` defaults to relaxed and `bsondump` writes canonical.

code

json · 8 lines
json
{
  "_id": {"$oid": "57e193d7a9cc81b4027498b5"},
  "qty": {"$numberInt": "7"},
  "views": {"$numberLong": "42"},
  "ratio": {"$numberDouble": "0.5"},
  "price": {"$numberDecimal": "19.99"},
  "at": {"$date": {"$numberLong": "1793664000000"}}
}

go deeper

for a junior

Recall that Extended JSON is BSON written as text, using dollar-prefixed wrappers for types JSON lacks, and that it comes in two modes: canonical for types, relaxed for reading.

for a middle

Explain the table: only Int32, Int64, finite doubles and in-range dates differ between modes. Then explain the parser rule that makes an Int64 of 42 come back as Int32.

for a senior

Separate what the format loses from what a pipeline loses. Know which tool writes which mode, and test a round trip by comparing BSON types on both sides instead of comparing values.

for a principal

Weigh readability against fidelity as a policy: relaxed for people and simple consumers, canonical for fixtures and type assertions. The specification's own warning about round trips belongs in that rule.

## Why Extended JSON exists **BSON** is a binary format with types JSON cannot name: ObjectId, Decimal128, Int32 against Int64, binary with a subtype, dates. **Extended JSON** is the string format that carries them, using **type wrappers**: JSON objects whose `$`-prefixed keys, such as `$oid` or `$numberDecimal`, encode a BSON type and value with JSON primitives only. Both modes conform to the JSON RFC, so any JSON parser can read the text, but only an Extended JSON parser turns the wrappers back into types. To a plain parser, `{"$numberDecimal": "19.99"}` is just an object with one string field. ## The two modes, row by row The specification defines one conversion table with two columns. Most rows say "same as canonical". These are the rows worth remembering: | BSON type | Canonical | Relaxed | |---|---|---| | Int32 | `{"$numberInt": "42"}` | `42` | | Int64 | `{"$numberLong": "42"}` | `42` | | Double, finite | `{"$numberDouble": "42.42"}` | `42.42` | | Double, non-finite | `{"$numberDouble": "NaN"}` | same | | Date, year 1970 to 9999 | `{"$date": {"$numberLong": "..."}}` | `{"$date": "2026-11-03T00:00:00Z"}` | | Date, any other year | `{"$date": {"$numberLong": "..."}}` | same | | Decimal128 | `{"$numberDecimal": "19.99"}` | same | | ObjectId | `{"$oid": "..."}` | same | | Binary | `{"$binary": {"base64": "...", "subType": "04"}}` | same | So **relaxed mode is not "plain JSON"**. An export in relaxed mode still contains `$oid`, `$date`, `$numberDecimal` and `$binary`. A date keeps its wrapper in both modes; only the value inside changes, from milliseconds to an ISO-8601 string. The types that truly lose their label are the three JSON-number rows. ## What a relaxed round trip changes The damage happens on the way back, under the specification's rules for parsing bare JSON numbers: 1. A number with a fraction or exponent should be read as a BSON **Double**. 2. An integer should be read as the **smallest BSON integer type** that holds it exactly. 3. An integer the parser cannot hold exactly must become a Double, even if that loses precision. Worked through: - **Int64 42** is written `42` and comes back as **Int32**. The value survives and the type does not. - **Int64 3000000000** is outside the Int32 range, so it comes back as Int64. - **Int32 42** comes back as Int32, unchanged. - **Double 42.0** is written `42.0` by a compliant generator, because the table requires a non-integer form, and it comes back as Double. A language-native JSON step that rewrites it as `42` turns it into an Int32. - **Int64 above 2^53** survives a parser that holds 64-bit integers, but a tool that holds every JSON number as an IEEE 754 double will silently round it. ## v1, strict mode and shell mode Before the specification, Extended JSON was a family of conflicting formats. The manual now calls that **Legacy Extended JSON (v1)**, with a *strict mode* that conformed to the JSON RFC and a *shell mode* aimed at MongoDB's internal parser and the old shell. Two wrapper shapes changed because they collided with query operators: `{"$binary": "...", "$type": "80"}` clashed with `$type`, and `{"$regex": ..., "$options": ...}` clashed with `$regex`. Their replacements are the nested `$binary` form and `$regularExpression`. Relaxed mode arrived with version 2 of the specification in 2017. Parsers may accept legacy input only when configured to, which is what `mongoimport --legacy` does. ## Which tool writes what - **`mongoexport`** defaults to relaxed; `--jsonFormat=canonical` switches it. - **`bsondump`** writes canonical, and its documentation calls it a diagnostic tool, not one for data ingestion. - **`mongoimport`** accepts either mode without being told which. - **Generators**: the specification says a default format should be relaxed. - **Parsers**: the specification says they must accept both by default. ## Limits shared by both modes Canonical mode is the better choice for type fidelity, but the specification is plain that Extended JSON is designed primarily for testing and human inspection and is "not designed to reliably round-trip BSON documents". Two reasons apply to both modes: JSON objects are unordered while BSON documents are ordered, and a `$`-prefixed field in real data cannot be told apart from a type wrapper with the same keys, because the format has no escape.

  • Why does relaxed mode keep the wrapper on Decimal128 when it drops the one on a double?
    A bare JSON non-integer should be parsed as a BSON Double, so writing a Decimal128 as `19.99` would turn it into a double on the way back. Keeping `$numberDecimal`, with the value as a string, preserves the type and the exact digits in both modes.
  • When is a relaxed-mode date not an ISO string?
    When its year is before 1970 or after 9999. Those dates use the canonical form in both modes, `{"$date": {"$numberLong": "..."}}`, because the specification limits the ISO form to a portable subset. Inside the range, the ISO form has at most millisecond precision and should omit zero fractional seconds.
  • How do the manual's v1 and v2 line up with the specification's versions?
    The manual's v1 is Legacy Extended JSON: strict and shell mode, with no single standard. Its v2 is the specification's version 2.0, from 2017, which added relaxed mode and kept canonical. The specification also had a short-lived 1.0 from early 2017, which is not the manual's v1.

  • Relaxed Extended JSON is plain JSON with no dollar-prefixed wrappers left in it.
  • Relaxed mode turns a Decimal128 into a JSON number.
  • Relaxed mode loses dates because they become bare strings.
  • Canonical Extended JSON guarantees a perfect BSON round trip in every case.
  • An Int64 always stays an Int64 when read back from a relaxed number.
  • bsondump and mongoexport both default to relaxed mode.
open as a page

Why can a MongoDB equality match on an embedded BSON document fail on field order, and what do duplicate keys do to the same assumption?

level: middleimportance: should knowfreq 25%

basics

~20 s

BSON documents are ordered, and MongoDB compares embedded documents including order, so {a:1,b:2} does not equal {b:2,a:1}. Match with dot notation. Duplicate keys are not forbidden by the format, but MongoDB does not support them.

open as a page

A nightly mongoexport is re-imported into a test MongoDB, and a Decimal128 price arrives as a double and a date as a string: how do you find the cause?

level: seniorimportance: should knowfreq 28%

basics

~20 s

Relaxed mongoexport is not the cause: it writes $numberDecimal and $date, and mongoimport restores both. Read the file. Bare values mean a CSV export or a flattening script stripped the wrappers, and untyped import then guessed double and string.

open as a page

A MongoDB filter by UUID finds nothing after one service moved to uuidRepresentation=standard while another still writes the old way: what does the setting decide, and how do you fix it?

level: seniorimportance: should knowfreq 22%

basics

~20 s

Drivers encode native UUIDs by uuidRepresentation: standard is subtype 4, while the Java, C# and Python legacy forms are subtype 3 with their own byte orders. Mismatches misread values or miss filters. Inspect stored bytes, set standard, and rewrite legacy values.

open as a page

What rule should a team set for moving MongoDB data between systems: BSON files, Extended JSON or an application format, and how do you decide per consumer?

level: principalimportance: should knowfreq 15%

basics

~20 s

Choose by consumer: BSON files between MongoDB deployments, Extended JSON for people and test fixtures, and a versioned application format for anything outside the team. Then enforce it: named flags, one UUID representation, and a type census on every pipeline.

open as a page

How is a MongoDB OP_MSG framed, what do the hello limits cap, and what does a 9.0 server check in its BSON?

level: middleimportance: nice to knowfreq 12%

basics

~10 s

An OP_MSG is a 16-byte header, 4 flag bytes and sections holding BSON documents. hello's maxMessageSizeBytes caps the whole message and maxBsonObjectSize each document. MongoDB 9.0 always validates incoming BSON structurally, whatever net.wireObjectCheck says.

open as a page