skip to content

Why does enumerating the same Laravel LazyCollection twice re-run its source, and what does remember() trade to avoid that?

level: seniorimportance: should knowfreq 25%

answer

  1. the source is a function, called per pass
  2. count() is a full pass
  3. remember() caches as it goes
  4. memory versus repeated reads or queries
  5. eager() caches everything up front

basics

~20 s

Each enumeration calls the generator function again, so a second pass re-reads the file or re-runs the query. remember() caches items as they are first pulled and replays them later, trading memory for fewer reads or queries.

solid answer

~40 s

A `LazyCollection` stores its source, and every call that iterates it — `count()`, `each()`, `all()`, a `foreach`, a `first()` — invokes the source function afresh. With a file source that is another full read; with a database-backed source it is another query. So `$rows->count()` followed by `$rows->each(...)` does the work twice. `remember()` returns a lazy collection that caches each key and value the first time it is pulled and serves them from that cache on later passes, continuing the original iterator only for items not yet seen. The price is memory: everything enumerated stays cached. `eager()` goes further and loads everything immediately into an array-backed lazy collection.

code

php · 19 lines
php
<?php

use Illuminate\Support\LazyCollection;

$reads = 0;

$rows = LazyCollection::make(function () use (&$reads) {
    $reads++;
    yield from ['DE', 'US', 'DE'];
});

$rows->count();
$rows->countBy()->all();
echo $reads; // 2 - the source function ran twice

$cached = $rows->remember();
$cached->count();
$cached->countBy()->all();
echo $reads; // 3 - one more run, then replayed from the cache

go deeper

for a junior

Remember that every pass over a lazy collection runs its source again, and remember() caches what was already read.

for a middle

Explain why the source function is re-invoked per enumeration, how remember() replays its cache and advances one iterator, and how eager() differs.

for a senior

Find double passes such as count() then each() in real jobs, and choose between one-pass restructuring and remember() by source cost and data size.

for a principal

Set expectations for streaming jobs on repeated reads, memory ceilings and source cost, so teams do not trade one outage for another.

## Where the second pass comes from A `LazyCollection` never stores its items. It stores a **source**, and its `getIterator()` builds a fresh iterator from that source every time: - if the source is an **array**, iteration is cheap and repeatable; - if the source is a **callable** (the generator function you passed to `make()`), it is **called again**, and a new generator starts from the beginning. That is also why `make()` refuses a `Generator` object: a function can be called any number of times, a generator object can be run only once. Every method that enumerates therefore starts over. These are common double passes: 1. `$lines->count()` to size a progress bar, then `$lines->each(...)` to process — the file is read twice. 2. `if ($rows->isNotEmpty()) { $rows->each(...); }` — the emptiness check starts one enumeration, the loop another. 3. Passing the same lazy collection to two report builders — each runs the source. With a database-streaming source, each pass is **another query**; with an HTTP-paginated source, **another round of API calls**; with a file, **another full read**. ## What counts as a pass Anything that asks the collection for its first item starts a new enumeration, even if it stops early: - `isEmpty()` / `isNotEmpty()` — create an iterator and check whether it has a first item; - `first()`, `contains()` — enumerate until they find a match; - `count()`, `sum()`, `avg()`, `reduce()`, `each()` — enumerate everything; - `all()`, `toArray()`, `toJson()`, `collect()`, `eager()` — enumerate everything into memory; - a plain `foreach` over the collection. Pipeline methods such as `map()` or `filter()` do **not** start a pass by themselves; the pass happens when something terminal consumes their result. ## remember(): cache as you go `remember()` wraps the source in a caching generator: ```php <?php $answers = $source->remember(); $answers->take(5)->all(); // reads items 1-5 from the source, caches them $answers->take(20)->all(); // replays 1-5 from cache, reads 6-20, caches them $answers->count(); // replays 1-20, reads the rest once ``` Inside, the method keeps **one** underlying iterator plus a `$cache` array of `[key, value]` pairs. On each pass it serves cached positions first and only advances the real iterator for positions it has not seen yet. The source is therefore read **at most once**, however many times you enumerate. ## The trade-off | Approach | Source runs | Memory held | |---|---|---| | plain `LazyCollection`, N passes | N times | about one item | | `remember()`, N passes | once, incrementally | every item enumerated so far | | `eager()` | once, immediately | every item, from the start | | `collect()` | once, immediately | every item, as an eager `Collection` | So `remember()` is the right tool when: - the source is **expensive** (a slow query, a remote API, a compressed file); - you will iterate **more than once**, often partially; - the total you end up enumerating **fits in memory**. It is the wrong tool for a single pass over a 4 GB log, where caching every line recreates the memory problem the lazy collection was meant to solve. ## Better than both: one pass Often the cleanest fix is to restructure so one enumeration does everything: - count while processing with `tapEach()` or inside the `each()` callback instead of a separate `count()`; - compute several aggregates in one `reduce()` whose carry is an array of totals; - replace an emptiness check plus loop with a single loop that sets a flag. ## Interview framing The coverage phrase to remember is **memory versus query count**. A plain lazy collection minimises memory and may repeat work; `remember()` minimises repeated work and spends memory on it; `eager()` and `collect()` spend all the memory up front. A strong answer names the double-pass trap (`count()` then `each()`), explains that the source function is re-invoked, and chooses between one-pass restructuring and `remember()` based on size and cost.

  • How does eager() differ from remember() on a Laravel LazyCollection?
    `eager()` calls `all()` straight away and returns a new lazy collection backed by that array, so the source runs once, completely, before you do anything else. `remember()` defers: it reads from the source only as far as enumeration asks, caching those items, so a partial `take(5)` touches only five.
  • How would you show progress while processing a lazy collection in Laravel without a separate count() pass?
    Count as you go: increment a counter inside `tapEach()` or the `each()` callback and report every N items, or use `withHeartbeat()` for time-based updates. If you need a total up front, get it from a cheaper source such as the file size or a separate count query, not by enumerating the lazy collection twice.

saying these in an interview costs you the question

  • Believes a LazyCollection caches items after the first enumeration.
  • Calls count() and then each() on a file-backed lazy collection without noticing two reads.
  • Uses remember() on a single pass over a multi-gigabyte file.
  • Thinks remember() restarts the source for items not yet cached.
  • Says eager() and remember() behave identically.