In Ragas FactualCorrectness, what do mode='precision' and mode='recall' change?

## What the metric does before it scores anything `FactualCorrectness` is not a similarity measure. It first asks the judge LLM to break the `response` into atomic claims, then to break the `reference` into atomic claims, and then it runs natural-language-inference checks between the two sets: is each response claim entailed by the reference, and is each reference claim entailed by the response. Those two directions produce two different counts, and `mode` selects which one you see. ## Precision mode The denominator is the number of claims in the response. The numerator is how many of them the reference supports. A response that says three things, two of which appear in the reference, scores two-thirds regardless of how much the reference also contained. The behaviour this creates is specific and often surprising: a model that produces a correct answer plus extra correct detail that simply is not in your reference gets marked down. That is not a bug, it is the metric doing its job — in a domain where unsupported additions are dangerous (medical, legal, financial), an extra claim you did not verify is exactly the thing to punish, and precision mode is how you make the number reflect that. ## Recall mode The denominator flips to the number of claims in the reference. The numerator is how many of them appear in the response. Now extras are free and omissions are expensive. This is the mode for questions whose answer has several required parts — a checklist, the steps of a procedure, all the eligibility criteria — where a fluent answer that mentions two of four requirements is a real failure that precision mode would score perfectly. ## Why f1 as the default is a reporting problem `mode="f1"` is the harmonic mean of the two and is the default. It is a reasonable single number and a poor diagnostic: a 0.6 could be a terse answer that omitted half the reference, or a verbose answer full of unverified additions, and those two failures have opposite fixes. Prompt for brevity and you make the second better and the first worse. Teams that only track f1 tend to oscillate. Report both directions during development and keep f1 for the summary row. ## Atomicity and coverage The `atomicity` and `coverage` arguments control the decomposition step — how aggressively a sentence is split into separate claims and how much of the source text the claims are required to account for. They matter because they move the denominators. Finer decomposition produces more, smaller claims, and the score changes even though neither your model nor your data did. The operational rule that follows: fix these settings when you establish a baseline and treat a change to them as a break in the time series, exactly as you would treat a change of judge model. A score comparison across different decomposition settings is not a comparison. ## Where it fits FactualCorrectness reads `reference`, so it lives entirely in the labelled offline suite — it cannot run on unlabelled production traffic. It is also one of the more expensive built-ins per sample, because a single score requires two decomposition calls plus entailment checks over the resulting claim sets, and both sets scale with answer length. Long-form answers cost noticeably more than short factual ones, which is worth knowing before you point it at a dataset of essay-length outputs. ## Reading a result In review, the useful framing is a sentence per direction. "Precision fell from 0.91 to 0.78 while recall held, so the new prompt is adding claims we cannot trace to the reference" is a diagnosis. "FactualCorrectness fell from 0.88 to 0.81" is a fire alarm with no address on it.