In Lucene, what is the difference between postings, doc values, and stored fields for the same field?

## The same value, written several ways Lucene never stores a field just once. When `IndexWriter` processes a document, each field is routed through as many independent write paths as its `FieldType` requests: an inverted path (postings), a columnar path (doc values), a row path (stored fields), plus optional norms, points and term vectors. Those paths land in different files inside the segment, are read by different APIs, and are tuned for opposite access patterns. Understanding which structure serves which operation is the single most useful mental model for reasoning about index size and about errors like *"unexpected docvalues type NONE"*. ## Postings: term to documents An analyzed field is broken into terms; the terms are sorted and written to a term dictionary (`.tim`), indexed by a compact automaton (`.tip`), and each term points at a postings list of document IDs (`.doc`), optionally with positions (`.pos`) and offsets/payloads (`.pay`) depending on the field's `IndexOptions`. This is the structure that makes search fast: for the term `kotlin`, Lucene jumps straight to a delta-compressed, skip-list-accelerated list of matching document IDs and their term frequencies, and never looks at non-matching documents. The crucial limitation is direction. Postings map term to documents. There is no cheap way to invert them at query time — asking "what is document 42's value for `status`?" from postings alone means enumerating terms until you find one whose list contains 42. That is why the second structure exists. ## Doc values: document to value Doc values are Lucene's columnar store, written per segment at flush time and never mutated afterwards. The types are `NUMERIC`, `SORTED_NUMERIC`, `BINARY`, `SORTED` and `SORTED_SET`. The two `SORTED*` variants do not store the raw bytes per document at all; they store a sorted set of distinct values once, and per document only an *ordinal* — a small integer pointing into that set. Comparing ordinals is what makes sorting and faceting on a string field cheap, because most of the work happens on packed integers rather than on byte strings. Doc values are consumed by sorting (`SortField`), faceting and aggregation, grouping, function/score computation, script field access, and join-style lookups. Since Lucene 7 they are exposed as a forward-only iterator (`DocValuesIterator` extends `DocIdSetIterator`), which cooperates well with the document-at-a-time query execution model but means code that needs random access has to re-acquire the iterator. Because doc values are packed and bit-compressed (delta, GCD and table encodings are applied where they fit), they are usually far smaller than the equivalent stored-field copy — but they are still a full extra column on disk. ## Stored fields: document to original bytes Stored fields are the row store. Values handed to a `StoredField` are written verbatim into chunks in `.fdt`, with an offset index in `.fdx` and metadata in `.fdm`, and the chunks are compressed. They are not searchable, not sortable, and not aggregatable — they exist purely so that after ranking you can hand the caller back what was indexed. Because reads happen only for the documents you return, stored fields tolerate high compression ratios in a way that doc values (read for every matching document) do not. ## Why not just one structure Consider a query that filters on `status`, sorts by `price`, and displays `title`. Postings on `status` cut the candidate set. Doc values on `price` feed the comparator for every surviving document. Stored fields on `title` are decompressed for the ten documents on page one. A single structure cannot be simultaneously good at set intersection, at dense sequential per-document reads, and at bulk-compressed random retrieval — so Lucene keeps three, and lets the schema decide which ones a field pays for. ## Practical consequences - A field indexed but without doc values will match queries and then fail at sort or facet time with an error naming the expected doc-values type. The fix is a reindex; doc values cannot be added to existing segments. - A field with doc values but not indexed can be sorted and aggregated but not efficiently searched. - A field neither indexed nor doc-valued but stored is display-only. - Enabling everything everywhere is the most common cause of a Lucene index that is several times larger than the source data. Every path you enable is a separate copy with its own encoding. The search engines built on Lucene expose these as per-field schema flags, which is why flipping one flag can change index size by tens of percent.