After a MongoDB service adopts client-side encryption, some queries throw errors and others quietly return wrong results, so how do you work out what the encrypted fields broke?
answer
- client refusal or server answer
- an allow-list, not a block-list
- ciphertext has no plaintext order
- old plaintext never matches new queries
basics
~20 sSeparate refusals from wrong answers. Automatic encryption rejects anything outside its allow-list on the client; wrong results come from the server comparing BinData, as with explicit operators, views, collation and plaintext left from before encryption.
solid answer
~40 sI first ask where the failure occurs. Under automatic encryption, the driver checks each command against an allow-list before sending, so `$regex` or a `null` comparison on an encrypted field, `$out` or `$merge`, `$text`, `$where` and `$jsonSchema` on any field, and unlisted commands all fail as client errors. Those are by design: the fix is to change the query. Silent wrong results are worse, because the server just compares `BinData`. Explicit encryption with `$gt` on a deterministic field succeeds with meaningless ordering; views and collation behave unpredictably; and a field added to `encryptedFieldsMap` later never matches its older plaintext documents. For Queryable Encryption I also check the structural limits: no secondary reads, unique or TTL indexes, or multi-document updates, and little diagnostic output.
code
javascript · 8 lines// Client configured for automatic Queryable Encryption;
// cardToken is equality-indexed, country is not encrypted.
db.customers.find({ cardToken: /^tok_/ }) // error: regex against an encrypted field
db.customers.find({ country: "DE", $where: "true" }) // error: $where, even on plaintext fields
db.customers.aggregate([{ $merge: "archive" }]) // error: $merge is unsupported
db.customers.updateMany({ country: "DE" },
{ $set: { tier: 2 } }) // error: multi-document update
db.customers.find({ cardToken: "tok_123" }) // fine, encrypted before it is sentgo deeper
Remember that an encrypted field only supports a small list of operations, and that a client configured for encryption refuses anything not on that list.
Know the operator, stage and expression lists for each scheme, the differences between them, and why comparing an encrypted field to null or a regular expression always fails.
Separate client refusals from silent wrong answers, and look for silent ones where no query analysis ran. Check data history and structural limits, and arrange application-side metrics before you need them.
Inventory the queries, reports and operational tools that touch a field before encrypting it. Most breakage is predictable, and the costly part is what you discover after the collection is already built.
## Two different failures Encrypted fields break existing queries in two ways, and the first step is telling them apart. - **Refusal.** A client configured for automatic encryption parses each command. If it would not work on ciphertext, the client returns an error, so nothing is sent. This is the system working as designed. - **Wrong answer.** The server only ever sees `BinData`. Where nothing stops a query reaching it, the server evaluates the query against those bytes. Depending on the operator, it returns an error or a result that does not reflect the plaintext. ## A diagnostic order 1. **Find where the error comes from.** An error from query analysis names an operation the client refuses. A KMS or key vault error is a key access problem, not a query problem. 2. **Confirm the field's configuration.** Is it encrypted, by which scheme, and with which algorithm or query type? Read `encryptedFields`, or the CSFLE schema, before guessing. 3. **Compare the operation with the allow-list.** The manual's supported operations pages list commands, operators, stages and expressions. Anything not listed fails. 4. **Confirm which mode took this path.** A client with `bypassAutoEncryption` or explicit encryption skips analysis, so it can produce a wrong answer where automatic mode would have refused. 5. **Check the data's history.** Look for documents written before encryption was turned on, and for writes that went around a configured client. ## What automatic encryption refuses | Area | CSFLE | Queryable Encryption | |---|---|---| | Value operators on the field | `$eq`, `$ne`, `$in`, `$nin`, `$and`, `$or`, `$not`, `$nor`, on deterministic only | The same plus `$expr`; range fields add `$lt`, `$lte`, `$gt`, `$gte` | | Update operators | `$set`, `$unset`, `$rename` | `$set`, `$unset` | | `$group` | May group on deterministic fields; no arithmetic accumulators on encrypted fields | Only on unencrypted fields | | `distinct` command | Allowed | Not on the supported list | Some rules apply on both sides: - Comparing an encrypted field to `null` or a regular expression always throws an error. - `$text`, `$where` and `$jsonSchema` fail even against unencrypted fields, purely because the client is configured for encryption. - Only `$cond`, `$eq`, `$ifNull`, `$in`, `$let`, `$literal`, `$ne` and `$switch` are accepted as aggregation expressions on encrypted fields, plus the `$encStr` family for Queryable Encryption string fields. - `$lookup` cannot join on an encrypted field, except that CSFLE allows a self-join. `$graphLookup` must read from the collection the aggregation runs against. - Commands and stages not on the list return an error, and neither list includes `$out` or `$merge`. Queryable Encryption also rejects `updateMany` and any `bulkWrite` with more than one update or delete. **Sorting** needs care. `$sort` appears on the supported stage lists, but no page promises plaintext order on an encrypted field, and ciphertext carries no such order. Treat a sort on an encrypted field as unavailable, and sort in the application after decryption. ## What quietly goes wrong - **Explicit comparison.** The manual's example is a deterministic `Salary` filtered with a greater-than. The operation succeeds, but it compares `BinData` values, so the result can differ from what the integers would give. - **Plaintext left behind.** If a previously plaintext field is added to `encryptedFieldsMap`, queries go out as `BinData`, and the old plaintext values never match. - **Collation.** Both schemes ignore collations on encrypted fields, and collation-sensitive queries can return incorrect results. - **Views.** The manual warns that a view whose pipeline or query touches an encrypted field may return incorrect results, so avoid that. The driver specification adds that automatic encryption does not support views, and `libmongocrypt` errors when collection info shows one, but do not lean on that check. - **Unique indexes.** Under Queryable Encryption and on random CSFLE fields, an index cannot enforce plaintext uniqueness, so duplicates accumulate without error. - **Metadata damage.** Modifying `__safeContent__` or the metadata collections produces incorrect results. ## Structural limits to rule out Queryable Encryption runs only on replica sets and sharded clusters, and it does not support secondary reads, so a `readPreference` of `secondary` does not fit it. Shard keys must not be encrypted fields, and the metadata collections must not be sharded. TTL and unique indexes are unsupported, and you should not index an encrypted field; the manual suggests an index on `__safeContent__` instead. Slowness has its own causes. A collection created implicitly lacks the required metadata and indexes, so create it explicitly. Metadata that has grown past 1 GB needs `compactStructuredEncryptionData`. Your usual tools also go dark. Queryable Encryption omits these operations from the slow query log and `system.profile`, redacts `currentOp` and `collStats`, and leaves them out of `$planCacheStats`. The manual therefore suggests measuring from the application with a third-party monitoring tool. Plan for this before the first incident, because it is hard to add during one.
- A field was added to the encrypted configuration last month, and old customers cannot be found by it. What is happening?Queries now send the value as `BinData`, and documents written before the change still hold plaintext, so they never match. Adding a field to `encryptedFieldsMap` affects only later inserts. Queryable Encryption has no in-place conversion, so migrating means inserting the documents through a configured client into a new collection.
- How do you investigate a slow query on a Queryable Encryption collection?Not from the usual sources, because it is left out of the slow query log and `system.profile`, and `currentOp` and `collStats` are redacted. Measure from the application with a third-party monitoring tool, check whether metadata collections exceed 1 GB and need compacting, and confirm the collection was created explicitly, since implicit creation lacks its metadata and indexes.
- A range comparison on a deterministic CSFLE field returns correct results because encryption preserves order.
- If a query does not return an error on encrypted data, its results must be correct.
- The operators $text and $where only fail when they touch an encrypted field.
- Adding a field to encryptedFieldsMap encrypts the existing documents' values too.
- A unique index on a Queryable Encryption field still enforces plaintext uniqueness.
- Secondary reads scale out Queryable Encryption queries just as they do normal queries.