How do NFC and NFD differ from NFKC and NFKD in unicodedata.normalize?

## Two axes, four forms `unicodedata.normalize(form, text)` accepts `"NFC"`, `"NFD"`, `"NFKC"` and `"NFKD"`. The names decode cleanly: - **D = decomposition**: split precomposed characters into a base character plus combining marks, then sort the marks into canonical order. - **C = composition**: decompose first, then recombine base plus marks into precomposed characters where they exist. - **K = compatibility** (K for *kompatibility*, since C was taken): before composing or decomposing, apply compatibility mappings as well as canonical ones. So NFD and NFC are the canonical pair, NFKD and NFKC the compatibility pair. ## Canonical equivalence: lossless Two strings are **canonically equivalent** when they represent exactly the same written character with no difference in meaning or appearance — a precomposed e-acute versus `e` plus COMBINING ACUTE ACCENT, or the same base carrying two marks stacked in either order. NFC and NFD both map every member of such a group onto one representative, and no information is lost: text normalized to NFD can be normalized back to NFC and you get the original. Nothing a reader would notice changes. ## Compatibility equivalence: lossy on purpose **Compatibility equivalence** is a deliberately weaker relation: characters that mean the same thing but differ in formatting. Unicode encodes many of these for round-tripping with older character sets: - typographic ligatures — `\ufb01` (a single character) folds to the two characters `fi` - Roman numerals — `\u2168` folds to `IX` - superscripts and subscripts fold to their plain digits - full-width and half-width forms fold to the plain ASCII forms - some spaces fold to a plain space, and some fractions expand The K forms perform these mappings; the non-K forms leave them alone. Note what that means for `unicodedata.is_normalized("NFC", text)`: a string containing a ligature is already valid NFC — NFC has no opinion about it at all. Compatibility folding is **not reversible**. Once `\u2168` has become `IX` you cannot tell it from a genuine `IX` the user typed, and re-normalizing will never bring it back. That is the single most important operational fact about the K forms. ## Which form do you actually use? **Store NFC.** It is what the web platform recommends for interchange, it is what most text on most systems already is, it is compact, and it preserves the user's input character for character. Data written in NFC survives a round trip through your service unchanged. **Use NFKC to build a matching key, never as the stored value.** When you want a search box to find a title whose author typed a ligature, or a lookup to treat a full-width identifier as equal to its ASCII form, compute an NFKC (often plus `str.casefold()`) key, index that, and keep the original text next to it. The moment you overwrite the stored value with its NFKC form you have edited the user's data — a display name, a legal name, a document title — with no way to undo it. **NFD has narrow, real uses.** It is the form in which combining marks are visible as separate characters, which is what you want when you inspect or filter marks — for example dropping every character whose `unicodedata.combining(ch)` is nonzero, or whose `unicodedata.category(ch)` is `Mn`, to build an accent-insensitive index key. Some platform APIs also hand you filenames in NFD, which is a classic source of "the file is right there but the lookup fails". **NFKD is rarest**: compatibility folding plus visible marks, useful mainly as the first step of an aggressive fold-everything search key. ## Cost and caching Normalization walks the string and consults the character database, so it is not free, but the implementation is C and there is a quick-check fast path: `unicodedata.is_normalized` can often answer without allocating, and normalizing text that is already in the requested form is cheap. On a hot path — say a triage bot classifying every inbound ticket title against a 92nd-percentile latency budget — normalize once when the record is created rather than on each comparison, and store the derived key so later requests do no Unicode work at all. ## A precise way to say it in an interview "C and D differ only in whether marks are composed; that difference is invisible to a reader and fully reversible. K is the real fork: it folds formatting variants like ligatures and Roman numerals, which is lossy. So NFC for storage, NFKC for a derived matching key, and never the other way round."