When do you use DeepEval's evaluate() instead of assert_test() in pytest?

## Two different jobs Both functions run metrics over test cases. The difference is what they do with the result. `assert_test(test_case=..., metrics=[...])` evaluates one case and raises `AssertionError` if any metric misses its threshold. It is designed to sit inside a pytest test, where raising is the contract. `evaluate(test_cases=[...], metrics=[...])` evaluates a list and returns an `EvaluationResult`. In deepeval 4.1.9 that object holds `test_results` (one `TestResult` per case, each with `success` and `metrics_data`), `confident_link` (a URL when the run was uploaded), and `test_run_id`. It never raises for a low score. ## The configuration surface `evaluate()` accepts four dataclass configs that `assert_test` sets for you: - `AsyncConfig(run_async=True, throttle_value=0, max_concurrent=20)` — whether to run asynchronously, how much to throttle, and how many evaluations may be in flight. `max_concurrent` is the knob you turn down when the judge provider starts returning rate-limit errors. - `DisplayConfig` — the progress indicator, whether results are printed, verbose metric output, which results to display, and an optional results folder for a timestamped JSON export of the run. - `CacheConfig(write_cache=True, use_cache=False)` — whether to write and whether to reuse cached metric results. - `ErrorConfig(ignore_errors=False, skip_on_missing_params=False)` — whether an errored metric or a case missing a required field aborts or is tolerated. It also takes `hyperparameters`, a dict of strings, numbers, or `Prompt` objects, recorded alongside the run so two runs can be compared by what changed, and `identifier` to label the run. ## Batching versus per-node parallelism This is the subtler distinction. `assert_test` evaluates one test case at a time; concurrency *across* cases comes from pytest running several nodes at once under `-n`. `evaluate()` owns the batching itself: give it 500 cases and it schedules them against `max_concurrent` without any test runner involved. So the two express parallelism at different layers, and mixing them — calling `evaluate()` on a large list inside a heavily parallel pytest session — multiplies concurrency in a way that reliably trips provider rate limits. ## Where each belongs Use `assert_test` when a human should be blocked from merging. Use `evaluate()` when you want a number: exploring in a notebook, sweeping a parameter, scoring a sample of production traffic on a schedule, or producing a report that a person reads and decides on. A useful hybrid is to run `evaluate()` in a scheduled job and gate CI on a much smaller `assert_test` suite, so the expensive breadth is decoupled from the merge path. You can, of course, use `evaluate()` inside pytest and assert on the aggregate yourself — for instance requiring that at least 90% of cases pass rather than every single one. That is a legitimate pattern when per-case strictness is too brittle, and it is only possible because `evaluate()` hands the results back instead of raising. ## Common mistakes Expecting `evaluate()` to fail a build: it will not, and a pipeline step that calls it and ignores the return value is green no matter how bad the scores are. Passing an enormous list without lowering `max_concurrent` and then blaming the provider for rate limiting. Assuming the run appears in a hosted dashboard without an API key configured — `confident_link` is `None` in that case. And treating the two functions as interchangeable spellings of the same thing when their failure semantics are opposites.