skip to content

Which Laravel LazyCollection methods stay lazy, and which quietly pull every item into memory, such as sortBy() or groupBy()?

level: seniorimportance: should knowfreq 32%

answer

  1. passthru() calls $this->collect()
  2. sort, groupBy, reverse, shuffle
  3. partition runs at call time
  4. countBy and unique keep state
  5. first, take, contains stop early

basics

~20 s

map, filter, reject, take, skip, chunk, values and tapEach stream one item at a time. Methods that need the whole set — sortBy, sort, groupBy, reverse, shuffle, split, median, partition — collect everything into an eager array first.

solid answer

~40 s

In `LazyCollection`, streaming methods such as `map`, `filter`, `reject`, `take`, `skip`, `takeUntil`, `chunk`, `values` and `tapEach` wrap the source in another generator and hold one item at a time. A group of methods is implemented through a protected `passthru()` helper that calls `$this->collect()` — `sort`, `sortBy`, `sortByDesc`, `sortKeys`, `groupBy`, `reverse`, `shuffle`, `split`, `merge`, `diff` and friends — so they still *return* a lazy collection, but the first enumeration loads every item into an eager `Collection`. `median()` and `mode()` collect at once, and `partition()` iterates the source as soon as you call it. `countBy()` and `unique()` stream but keep state proportional to distinct values. Terminal methods (`each`, `count`, `sum`, `reduce`, `all`) enumerate everything in one pass.

code

php · 17 lines
php
<?php

use Illuminate\Support\Facades\File;

$lines = File::lines(storage_path('logs/access.log'));

// Collects all lines on first enumeration (passthru -> collect()):
$bad = $lines->sortBy(fn ($l) => $l)->take(5)->all();

// Streams: aggregate first, then sort the small result
$good = $lines
    ->map(fn ($l) => explode(',', $l)[2] ?? null)
    ->filter()
    ->countBy()
    ->sortDesc()
    ->take(5)
    ->all();

go deeper

for a junior

Remember that sorting and grouping a lazy collection load every item, while map, filter and take stream.

for a middle

Explain the passthru() helper, which methods it covers, and why a method that must see the last item before its first output cannot stream.

for a senior

Review pipelines for hidden collecting steps and stateful methods on high-cardinality data, and reorder them to aggregate before sorting.

for a principal

Decide when a pipeline's memory profile is a correctness requirement worth tests or runtime guards, not just a performance hope.

## Why the question matters A lazy pipeline saves memory only while **every step streams**. One step that needs all the data defeats the whole design, and because those steps still return a `LazyCollection`, nothing in the types warns you. Interviewers ask this to see whether a candidate has read past the "lazy collections use less memory" headline. ## Streaming methods These return a new `LazyCollection` whose generator reads the previous one item by item: - `map`, `filter`, `reject`, `mapWithKeys`, `pluck`, `keyBy`, `values`, `keys` - `take` (positive limit), `skip`, `takeUntil`, `takeWhile`, `skipUntil`, `skipWhile`, `takeUntilTimeout` - `chunk`, `chunkWhile`, `sliding`, `tapEach`, `throttle`, `withHeartbeat` - `slice` with a non-negative offset and length (it becomes `skip` + `take`) Their memory cost is one item, or one chunk for `chunk($size)`. ## Streaming, but with state | Method | What it keeps | Grows with | |---|---|---| | `unique()` | every value seen so far | number of distinct values | | `countBy()` | a counter per group, yielded at the end | number of groups | | `take(-n)` | a ring buffer of the last `n` items | `n` | These are safe when the number of distinct values is small (countries, status codes) and dangerous when it is not (unique request IDs). ## Methods that collect everything Inside `LazyCollection` the source contains a helper: ```php <?php protected function passthru($method, array $params) { return new static(function () use ($method, $params) { yield from $this->collect()->$method(...$params); }); } ``` `collect()` enumerates the entire source into an eager `Collection`. Methods routed through it include: - ordering: `sort`, `sortDesc`, `sortBy`, `sortByDesc`, `sortKeys`, `sortKeysDesc`, `sortKeysUsing`, `reverse`, `shuffle` - grouping and splitting: `groupBy`, `mapToDictionary`, `split` - set operations: `merge`, `mergeRecursive`, `union`, `diff*`, `intersect*`, `crossJoin`, `replaceRecursive`, `except`, `duplicates` - `slice` with a **negative** offset or length, `pad` with a negative size, `dot`, `undot` The collecting is **deferred**: `$lazy->sortBy('score')` returns immediately, and the full load happens when the result is first enumerated. Two more collect at a different moment: 1. **`median()` and `mode()`** call `$this->collect()` directly and return a number, so they load everything the moment you call them. 2. **`partition()`** is not overridden for lazy collections. The shared implementation iterates the source immediately and builds both halves as arrays, so a lazy `partition()` holds every item in memory right away. ## Terminal methods `each`, `count`, `sum`, `avg`, `reduce`, `last`, `all`, `toArray`, `toJson` and `collect` enumerate everything. `sum`, `avg`, `count` and `reduce` do it in constant memory (unless your reducer accumulates); `all`, `toArray` and `collect` build the whole array by definition. ## Short-circuiting methods `first()`, `contains()` with a callback or value, `take()` and `takeUntil()` stop pulling as soon as they have their answer, so on a huge source they may touch only a few items. That makes them the cheapest checks — and means any cleanup code after the source loop may never run. ## A review checklist When a pull request adds a lazy pipeline over large data, check: 1. no `sort*`, `groupBy`, `reverse`, `shuffle`, `split`, `partition`, `median` or `mode` on the **raw** stream; 2. no `all()`, `toArray()` or `collect()` before the aggregation step; 3. `unique()` and `countBy()` only on fields with bounded cardinality; 4. a single enumeration — no `count()` or `isEmpty()` followed by a second pass; 5. any expensive step placed after `filter()`, so it runs on fewer items. ## How to reason about a new method 1. Can it produce its **first** output item before seeing the **last** input item? If not (sorting, reversing, grouping), it must collect. 2. Does it keep something for every distinct input? Then memory grows with cardinality. 3. When in doubt, read the method in `LazyCollection.php`: a body that returns `new static(function () { foreach … yield … })` streams; a body calling `passthru()` or `collect()` does not. Reordering a pipeline often rescues it: aggregate first (`countBy`) and sort the small result, rather than sorting millions of raw rows.

  • Why does partition() on a Laravel LazyCollection load everything as soon as it is called?
    `LazyCollection` does not override `partition()`, so it uses the shared `EnumeratesValues` version, which passes the iterator to `Arr::partition()`. That loops over every item immediately and returns two arrays, wrapped as collections, so both halves are fully in memory before you enumerate either.
  • Is unique() safe on a lazy stream of 100 million request log lines in Laravel?
    It streams, but it remembers every distinct value it has yielded so it can skip repeats. On a low-cardinality field such as country that is a few hundred entries; on request IDs it grows toward the full dataset and defeats the lazy design.

saying these in an interview costs you the question

  • Assumes any method returning a LazyCollection keeps memory flat.
  • Sorts millions of raw lines before aggregating them.
  • Believes sortBy() on a lazy collection sorts in streaming batches.
  • Thinks unique() on a lazy collection uses constant memory whatever the data.
  • Says partition() splits a lazy stream into two lazy streams.