In ASP.NET Core 10, how do returning a lazy IEnumerable<T>, a List<T> from ToListAsync and an IAsyncEnumerable<T> differ for a large query result?
answer
- serializer enumerates synchronously
- ToListAsync buffers everything first
- System.Text.Json streams IAsyncEnumerable
- Newtonsoft and XML formatters buffer
- MaxIAsyncEnumerableBufferLimit = 8192
basics
~20 sA lazy IEnumerable<T> backed by a query is enumerated synchronously during serialization, blocking a thread on I/O. ToListAsync is async but holds the whole result in memory. IAsyncEnumerable<T> lets System.Text.Json stream items as they arrive. Paginate anyway.
solid answer
~40 sWhen an endpoint returns a **lazy `IEnumerable<T>`**, for example an unmaterialised EF Core query, the JSON serializer enumerates it **synchronously** while writing the response, so the database I/O runs on a blocked thread: sync-over-async in disguise. **`await query.ToListAsync(ct)`** fixes the blocking, but the whole result is materialised in memory before the first byte is written, which for large results means big allocations and a late first byte. **`IAsyncEnumerable<T>`** (for example `query.AsAsyncEnumerable()`) lets System.Text.Json **stream** items to the response as they arrive, asynchronously. That holds for Minimal APIs and for MVC with the default System.Text.Json formatter; with the Newtonsoft.Json or XML formatters, MVC buffers an `IAsyncEnumerable<T>` first, up to `MvcOptions.MaxIAsyncEnumerableBufferLimit` (8192 entries). None of this replaces **pagination**: a client that needs 200 rows should not receive 200,000.
code
csharp · 12 lines// blocks: the serializer enumerates the query synchronously
app.MapGet("/shipments/lazy", (FreightDb db) =>
db.Shipments.Where(s => s.Status == ShipmentStatus.InTransit));
// async, but buffers every row before writing
app.MapGet("/shipments/page", async (FreightDb db, int page, CancellationToken ct) =>
await db.Shipments.OrderBy(s => s.Id)
.Skip(page * 100).Take(100).ToListAsync(ct));
// streams rows to the response as they are read
app.MapGet("/shipments/export", (FreightDb db) =>
db.Shipments.OrderBy(s => s.Id).AsAsyncEnumerable());go deeper
Recall that endpoints should return materialised lists from async queries or IAsyncEnumerable<T>, never a lazy query, and should paginate.
Explain synchronous enumeration by the serializer, the memory cost of ToListAsync, and when IAsyncEnumerable<T> streams versus buffers.
Choose per endpoint between bounded pages and streaming, accounting for error handling after the first byte and connection hold time.
Set API rules on result sizes, pagination and export endpoints so that no service ships unbounded collections.
## Three ways to return the same rows The freight dashboard has an endpoint that returns shipments for a date range. The result can be a few rows or tens of thousands. How the handler returns them decides whether a thread blocks, how much memory the request holds, and when the client sees the first byte. | Return shape | How the rows are read | Memory | Thread | |---|---|---|---| | Lazy `IEnumerable<T>` (unmaterialised query) | Serializer enumerates **synchronously** while writing | Streams | **Blocked on database I/O** | | `List<T>` from `await ToListAsync(ct)` | Read asynchronously, then serialized | **Whole result in memory** | Free while reading | | `IAsyncEnumerable<T>` | Serializer awaits each item and writes as it goes | Streams | Free while waiting | ## The lazy IEnumerable<T> trap An EF Core `IQueryable<T>` or any deferred `IEnumerable<T>` does no work until it is enumerated. If the handler returns it as `IEnumerable<T>`, the enumeration happens inside the response serializer, through the synchronous `IEnumerable<T>` interface. The ASP.NET Core best-practices page states the consequence: synchronous collection iteration by the serializer, blocking calls and potential thread-pool starvation. It is sync-over-async that never shows `.Result` in your code, which is why it survives code review. ## Buffering with ToListAsync `await query.ToListAsync(ct)` reads the rows asynchronously, so no thread blocks. The cost moves to memory and latency: - every row is materialised before serialization starts, so the client waits for the full query before receiving anything; - a large list means large allocations; arrays of 85,000 bytes or more land on the large object heap, which only full collections clean up; - many concurrent large requests multiply that memory. For small, bounded results this is the simplest correct choice. ## Streaming with IAsyncEnumerable<T> Returning `IAsyncEnumerable<T>` hands the serializer an asynchronous sequence. System.Text.Json awaits each item, writes it into the response and flushes as its buffer fills, so the thread is free between items and memory stays bounded by the buffer, not the result size. - **Minimal APIs** serialize the return value with System.Text.Json, so `IAsyncEnumerable<T>` streams. - **MVC with the default System.Text.Json formatter** streams too. - **MVC with Newtonsoft.Json or the XML formatters** buffers instead: the formatter reads the whole sequence into a list first and throws `InvalidOperationException` once it passes `MvcOptions.MaxIAsyncEnumerableBufferLimit`, which defaults to **8192** entries. Streaming has trade-offs. Once the first item is written, the status code and headers are sent, so a database error halfway through cannot become a clean 500 with a ProblemDetails body; the client sees a truncated response. The database connection also stays open for the whole response, held for as long as the slowest client takes to read. ## Pagination still comes first The guidance is to return large collections across multiple smaller pages. Streaming makes a large response cheaper; it does not make it a good API. For the dashboard: 1. add `page` and `pageSize` with an upper bound and return one page with `ToListAsync`; 2. use `IAsyncEnumerable<T>` for exports and feeds where the client really wants everything; 3. never return a lazy query as `IEnumerable<T>`. ## Choosing for the freight dashboard - **Shipment list on the main screen:** bounded pages with `ToListAsync`; clean error responses matter and pages are small. - **Nightly CSV or JSON export for a carrier:** `IAsyncEnumerable<T>`, so memory stays flat however large the export grows. - **Report summary built from many rows:** aggregate in the database and return the small result, rather than streaming rows to aggregate in memory. - **Any endpoint returning a query:** materialise it or stream it, but never return it as a lazy `IEnumerable<T>`. ## How to check what you shipped Under a load test, an endpoint returning a lazy query shows the starvation signature (low CPU, creeping thread count). A `ToListAsync` endpoint with huge pages shows memory spikes and frequent full collections. A streaming endpoint shows a fast first byte and flat memory, at the cost of longer-held connections.
- What happens if the database fails halfway through a streamed IAsyncEnumerable<T> response?By then the status line and headers have been sent with the first items, so the server cannot change the response to a 500 or write a ProblemDetails body. The exception aborts the response and the client receives truncated JSON. Endpoints where clean error responses matter should buffer a bounded page instead of streaming.
- Why does MVC with Newtonsoft.Json not stream IAsyncEnumerable<T>?Newtonsoft.Json serializes synchronously, so the Newtonsoft.Json formatter first reads the async sequence into a list and then serializes that collection. The buffer is capped by `MvcOptions.MaxIAsyncEnumerableBufferLimit`, 8192 entries by default, and passing it throws `InvalidOperationException`. Only the System.Text.Json path streams.
- Returning an EF Core query as IEnumerable<T> is fine because the serializer enumerates it asynchronously.
- ToListAsync streams rows to the client as they are read.
- IAsyncEnumerable<T> streams in MVC regardless of which output formatter is selected.
- Streaming a response lets the server still return a clean 500 if the query fails midway.
- With IAsyncEnumerable<T> available, pagination is no longer necessary.