Elasticsearch

A data stream is one name in front of an ordered set of hidden backing indices. Writes are append-only and always land in the newest backing index; searches and rollover target the stream name, not any single index.

Elasticsearch's dynamic setting decides what happens to an unmapped field: true indexes it and adds it to the mapping, runtime adds it as a query-time field only, false ignores it but keeps it in _source, strict rejects the document.

text is analyzed into tokens for full-text search and cannot be sorted or aggregated by default. keyword stores the value verbatim as one term, which is what filters, sorts and aggregations need.

An index template attaches settings, mappings and aliases to any index whose name matches its index_patterns. It is applied once, at index creation time; indices that already exist keep whatever configuration they were created with.

An alias is a second name that points at one or more indices. Applications talk to the alias so the physical index behind it can be rebuilt, versioned and swapped atomically, with no client change and no downtime.

Call POST /_analyze with an analyzer name and text to test any built-in analyzer, or POST /<index>/_analyze with a field and text to run that field's configured analyzer. Add explain: true to see each pipeline stage's output.

The standard analyzer splits on Unicode word boundaries and lowercases; simple splits at any non-letter and lowercases, so digits disappear; whitespace splits only on spaces and keeps case and punctuation; keyword emits the whole input as one token.

A custom analyzer is zero or more character filters, exactly one tokenizer, then zero or more token filters. Character filters rewrite the raw string, the tokenizer splits it into tokens, and token filters transform that token stream in listed order.

Both. Elasticsearch analyzes a text field's value when the document is indexed and stores the resulting terms; it analyzes the query string of a full-text query at search time. A hit requires the two term sets to overlap.

edge_ngram emits only prefixes anchored at the start, so Quick with grams 2 to 4 gives Qu, Qui, Quic. ngram emits substrings at every position, producing far more terms and a much larger index for infix matching.

Both are mandatory and select exactly the same documents. A must clause runs in query context and adds a relevance contribution to _score; a filter clause runs in filter context, computes no score, and its result set can be reused from the node query cache.

A match query runs the string through the field's search analyzer, turning it into terms, then builds a boolean query over those terms. By default a document matching any single term is a hit.

Because from + size exceeds index.max_result_window, which defaults to 10,000. Deep paging makes every shard build a top-(from+size) list that the coordinating node merges and then throws almost all of away, so Elasticsearch caps the depth instead.

A term query looks up the exact bytes you supply with no analysis, while a text field stores analyzed tokens that are split and lowercased. Searching for New York finds no such token. Use match, or a keyword sub-field.

Only when minimum_should_match requires it. If the bool query has at least one should clause and no must or filter clauses, minimum_should_match defaults to 1; otherwise it defaults to 0, so should clauses only add to _score.

A terms aggregation groups matching documents by the distinct values of a field and returns one bucket per value with a doc_count. size caps how many buckets come back, defaulting to 10, ordered by descending count.

Each shard independently returns only its own top candidate terms, and the coordinating node sums those partial lists. A term ranked low on one shard contributes nothing from it, so counts can be undercounted or the term missed entirely.

The cardinality aggregation estimates distinct counts with a HyperLogLog++ sketch of fixed size instead of tracking every value, so memory stays bounded as cardinality grows. precision_threshold trades memory for accuracy: higher means near-exact counts up to a larger unique count.

The percentiles aggregation summarizes the value distribution in a bounded TDigest sketch rather than sorting every value, so results are approximate. TDigest is most accurate at extreme percentiles and least accurate near the median, and per-shard sketches are merged.

Analyzed text fields have no doc_values, so aggregating on them makes Elasticsearch uninvert the index into heap-resident fielddata, whose size grows with token count. That risks the heap, so it is disabled by default; aggregate on a keyword sub-field instead.

A boost is a multiplier on one query clause's score contribution: title^3 inside a multi_match triples what a title match adds, and any query's boost parameter does the same. Boosts are relative weights, and filter context ignores them.

The field must be typed dense_vector with a fixed dimension count, left indexed so an HNSW graph is built, and given a similarity metric (cosine, dot_product, l2_norm or max_inner_product) that matches the embedding model.

Call GET /<index>/_explain/<id> with the query body, or add "explain": true to a search. Elasticsearch returns a nested tree breaking the score into per-clause weights with the idf and tf formulas and the actual N, n, freq, dl and avgdl values used.

score_mode combines the results of the entries in the functions array with each other; boost_mode combines that single combined function score with the score of the inner query. Both default to multiply, and boost_mode replace discards the query score entirely.

k is how many nearest neighbours the search returns; num_candidates is how many candidates each shard explores in the HNSW graph before picking its best k. Raising num_candidates buys recall at the cost of latency.

Green: every primary and replica shard is assigned. Yellow: all primaries are assigned but at least one replica is not. Red: at least one primary is unassigned, so part of the data is missing from search results and writes to that shard fail.

Elasticsearch search is near-real-time, not real-time. Indexing puts the document in an in-memory buffer and the translog; it only becomes searchable when a refresh turns that buffer into a new searchable Lucene segment, by default about once a second.

Elasticsearch hashes the document's routing value, which defaults to its _id, with Murmur3 and reduces that hash modulo the index's routing-shard count to pick exactly one primary shard. Placement is pure arithmetic, computed on any node, never looked up.

In the query phase each shard runs the search locally and returns only document ids plus sort values or scores; the coordinating node merges these into one globally sorted list. The fetch phase then retrieves the _source of just the winning documents.

Every shard is a separate Lucene index with fixed overhead in heap, file handles, merge work and cluster-state metadata, and every search becomes a task per shard that must be scheduled and merged. Those per-shard costs dominate once shards are small, which is why guidance targets tens of gigabytes per shard.