skip to content

A Dart log viewer stores logLines.where(isError).map(parseEntry) in a variable and parseEntry runs twice per error line; why, and how do you fix it?

level: seniorimportance: should knowfreq 42%

answer

  1. the variable holds a recipe
  2. every iteration is a new pass
  3. results are not cached
  4. side effects repeat per pass
  5. toList once at the boundary

basics

~10 s

A Dart where/map chain is a lazy Iterable that recomputes on every iteration, so two consumers mean two passes and two parses per error line. Materialise once with toList() and reuse the list.

solid answer

~50 s

`where` and `map` return lazy `Iterable`s that store the source and the callbacks, not results. Every operation that needs elements — a `for` loop, `toList()`, `first`, `isEmpty`, `length` — starts a **new pass** over `logLines`, and mapped values are not cached, so a loop plus a `toList()` parses each error line twice and repeats any logging inside the callbacks. What runs per pass depends on the operation: a mapped iterable may skip its function for `length` or `elementAt`, but the `where` test still has to run. Passes also see later changes to `logLines`. The fix is to call `.toList()` once where the pipeline is complete and hand the `List` to every consumer. Stay lazy only when the result is consumed once or a consumer may stop early, and keep `map`/`where` callbacks free of side effects.

code

dart · 24 lines
dart
var parses = 0;

String parseEntry(String line) {
  parses++;
  return line.substring(line.indexOf(' ') + 1);
}

void main() {
  final logLines = ['INFO boot', 'ERROR disk', 'ERROR net'];

  final lazy = logLines.where((l) => l.startsWith('ERROR')).map(parseEntry);
  lazy.toList();
  lazy.toList();
  print(parses); // 4: two passes, two error lines each

  parses = 0;
  final stored = logLines
      .where((l) => l.startsWith('ERROR'))
      .map(parseEntry)
      .toList();
  stored.length;
  stored.toList();
  print(parses); // 2: parsed once, then reused
}

go deeper

for a junior

Remember that a variable holding a where or map result is not a list of results, and each loop over it recomputes them.

for a middle

Explain which operations start a new pass, why callbacks re-run, and how toList() changes the cost and the snapshot behaviour.

for a senior

Diagnose duplicated work and side effects from lazy chains, place materialisation deliberately, and keep callbacks pure in reviewed code.

for a principal

Set conventions for return types and data boundaries so laziness is chosen on purpose rather than discovered in profiling.

## The bug A log viewer filters and parses lines once, then uses the result in several places: ```dart final errors = logLines .where((line) => line.contains('ERROR')) .map(parseEntry); // parseEntry is expensive and logs 'parsing' if (errors.isEmpty) return; for (final entry in errors) { showEntry(entry); } uploadAll(errors.toList()); ``` The developer expects each line to be tested once and each error parsed once. Instead, the `where` test runs over the lines several times and `parseEntry` runs **twice per error line**, and the log fills with duplicate `parsing` messages. ## Why it happens `where` and `map` return **lazy `Iterable`s**. The Dart docs are explicit: methods that return another `Iterable` "will iterate the original (as necessary) every time the returned iterable is iterated, and not before", and the converted elements **are not cached**. The variable `errors` holds a *recipe*, not results. Each operation that needs elements starts a **new pass** over `logLines` through the whole chain: 1. `errors.isEmpty` starts a pass and stops at the first error line. 2. The `for` loop starts a full pass: every line is tested, every error line parsed. 3. `errors.toList()` starts another full pass: tested and parsed again. What each pass evaluates depends on the operation. The docs allow a mapped iterable to skip `toElement` where the result is not needed — `length` and `isEmpty` on a mapped iterable can be answered from the source, and `elementAt` may call the function once — but the `where` test must still run to know which lines qualify. So `length` on this chain re-tests every line even though it may parse none. ## Symptoms in real code - **Repeated side effects**: logging, analytics calls or counters inside `map` or `where` fire once per pass. - **Repeated cost**: parsing, regex matching or date formatting redone for every consumer. - **Inconsistent results**: if `logLines` is appended to between passes, the second pass sees the new lines; if the callback reads mutable state or the clock, passes can disagree. - **A Flutter variant**: a `State` field or a `build` method that keeps an `Iterable` from `map` and iterates it on every rebuild re-runs the callback each time. - **The opposite bug**: a `map` used only for its side effects, whose result is never iterated, **never runs at all**. ## The fix Materialise once, at the boundary where the pipeline is complete: ```dart final errors = logLines .where((line) => line.contains('ERROR')) .map(parseEntry) .toList(); // one pass, results stored if (errors.isEmpty) return; for (final entry in errors) { showEntry(entry); } uploadAll(errors); ``` Now each line is tested once and each error parsed once, `errors.length` is constant time, and later changes to `logLines` no longer leak into the result. ## When to stay lazy Laziness is still the right default when: - the result is consumed **exactly once**, for example by `join`, `fold` or a single `for` loop; - a consumer may **stop early** — `first`, `any`, `firstOrNull`, `take(20)` — so the rest of the input is never processed; - the source is huge or unbounded and a full list would waste memory. | Situation | Keep `Iterable` | Call `toList()` | |---|---|---| | one consumer, one pass | yes | no need | | several consumers or repeated reads | no | yes | | side effects in the callbacks | no | yes, or move the effects into a loop | | early exit likely | yes | would do wasted work | | snapshot must not change with the source | no | yes | ## Review checklist 1. Look for `Iterable` variables, fields and return values that are read more than once. 2. Keep `map` and `where` callbacks **pure**; put side effects in an explicit `for` loop. 3. Name the boundary: a function that returns `Iterable` promises laziness; one that returns `List` promises stored results. Choose deliberately. 4. When debugging, a `print` inside the callback shows each pass — an easy way to confirm the diagnosis before and after `toList()`.

  • If logLines gets a new ERROR line after the lazy Dart chain is created, does the next iteration see it?
    Yes. The chain keeps a reference to `logLines` and reads it again on every pass, so an iteration that starts after the append includes the new line. A list produced by `toList()` is a snapshot and does not change. Modifying `logLines` during an iteration throws a `ConcurrentModificationError` instead.
  • Does calling length on logLines.map(parseEntry) in Dart run parseEntry?
    Not necessarily. The docs let a mapped iterable skip its function when the result is not needed, and the SDK answers `length` on a mapped iterable from the source. Put a `where` in front, though, and `length` must run the `where` test on every line, because counting matches needs it.
  • When is keeping the Dart pipeline lazy the better choice?
    When the result is consumed exactly once, or a consumer may stop early with `first`, `any`, `firstOrNull` or `take`, so later lines are never processed. Materialising there would do work that is thrown away and hold every result in memory.

A lazy Iterable is a recipe card, not a cooked dish: every guest who asks for the dish makes the kitchen cook it again from the raw ingredients, including whatever was added to the pantry since. toList() cooks once and puts the dish on the table.

saying these in an interview costs you the question

  • A where/map chain computes its results once and caches them.
  • Storing the chain in a final variable prevents recomputation.
  • Side effects in a map callback run exactly once per element.
  • A lazy chain ignores changes made to its source list.
  • Calling toList() is always wasteful and should be avoided.