LlamaIndex

When the control flow is known in advance. A prebuilt agent pays an LLM turn to decide every step; a hand-written Workflow encodes the sequence in typed events and steps, making it cheaper, deterministic and testable — at the price of owning the loop yourself.

Freeze an eval set, then measure retrieval and generation separately: RetrieverEvaluator with hit_rate and MRR over labelled query/node pairs, and BatchEvalRunner with the faithfulness and correctness evaluators over answers. Hold the judge model fixed, and report cost and latency next to quality.

Move when the index stops fitting one process: the corpus exceeds memory, several replicas must share it, writes must be incremental rather than a whole-file rewrite, or durability and concurrent access become requirements. Until then SimpleVectorStore is faster to iterate on.

Decide by how much of the corpus is layout-dependent. Built-in readers extract a raw text stream and lose tables, column order and scanned pages; LlamaParse is a hosted parser returning structured markdown, at a per-page price, added latency, and data leaving your environment.

Attribute the time first: measure retriever.retrieve() alone against the full query() to see whether retrieval, postprocessing or synthesis dominates. Then trade deliberately — candidate pool size, reranker placement, response mode and streaming each buy latency back at a different quality cost.