skip to content

Why can a Serilog-based .NET app lose its last log events on exit or after a crash, and how does Log.CloseAndFlushAsync() prevent it?

level: seniorimportance: must knowfreq 55%

answer

  1. some sinks buffer on a background loop
  2. BatchingOptions: 1000 events, 2 seconds
  3. Logger is IAsyncDisposable
  4. swaps Log.Logger for Logger.None
  5. await it in finally

basics

~20 s

Batched, buffered and async sinks hold events in memory and write them later, so exiting without disposing the logger abandons them. Log.CloseAndFlushAsync() disposes the static logger, draining those sinks, and resets Log.Logger; await it in finally.

solid answer

~40 s

Not every sink writes synchronously. A batched sink - any `IBatchedLogEventSink` registered with `WriteTo.Sink(sink, batchingOptions)` - queues events and emits them on a background task in batches of up to `BatchSizeLimit` (1000) or every `BufferingTimeLimit` (2 seconds). A file sink with `buffered: true` holds data in a stream buffer, and a `WriteTo.Async(...)` wrapper queues events too. If the process ends before those buffers are drained, the last events - often the crash itself - are lost. `Log.CloseAndFlushAsync()` swaps `Log.Logger` for the silent `Logger.None` and disposes the old logger; disposal disposes every sink, and the batching sink waits for its loop to emit what is queued. Put it in a `finally` around the app's run, after a `catch` that logs the exception at `Fatal`.

go deeper

for a junior

Remember the pattern: try, catch with Log.Fatal, finally with await Log.CloseAndFlushAsync(), so the last messages reach the file.

for a middle

Explain which sinks hold events in memory - batched, buffered file, async wrapper - and that disposing the logger disposes and drains its sinks.

for a senior

Reason about BatchingOptions defaults, QueueLimit drops, retry time while a destination is down, and the cases flushing cannot save, such as a hard kill.

for a principal

Weigh buffering for throughput against the window of events at risk on a crash, and choose batch timings and queue limits per sink accordingly.

## Where events wait A `Log.Information(...)` call returns as soon as the event has been handed to every sink. What a sink does with it next varies: | Sink style | Example | When it reaches its destination | |---|---|---| | synchronous | `WriteTo.Console()` | before the call returns | | unbuffered file | `WriteTo.File(path)` (default `buffered: false`) | flushed after each event | | buffered file | `WriteTo.File(path, buffered: true)` | when the stream buffer fills, on `flushToDiskInterval`, or on dispose | | batched | an `IBatchedLogEventSink` via `WriteTo.Sink(sink, new BatchingOptions())` | on a background loop, in batches | | async wrapper | `WriteTo.Async(a => a.File(...))` from `Serilog.Sinks.Async` | on a background worker thread | Network sinks that ship events to a log server are usually batched, so a production setup almost always has something in memory at any moment. ## How Serilog 4 batching works The core `Serilog` package contains `IBatchedLogEventSink` (with `EmitBatchAsync` and an optional `OnEmptyBatchAsync`) and a `BatchingSink` that wraps it. Its `BatchingOptions` defaults, from the source: - `BatchSizeLimit` = **1000** events per batch; - `BufferingTimeLimit` = **2 seconds** maximum wait before a partial batch is sent; - `EagerlyEmitFirstEvent` = **true**, so the first event goes out at once; - `QueueLimit` = **100000**; beyond it, new events are discarded; - `RetryTimeLimit` = **10 minutes** of retrying failed batches. `Emit` only writes the event into an in-memory channel. A background task reads the channel and calls `EmitBatchAsync`. If the process exits, that task and its queue simply vanish. ## What disposal does `Serilog.Core.Logger` implements `IDisposable` and `IAsyncDisposable`. Disposing it disposes every sink it owns. For the batching sink, disposal signals shutdown, waits for the run loop to finish - the loop emits what is already queued - and then disposes the inner sink. A buffered file sink flushes its stream when disposed. The static helper does this for `Log.Logger`: 1. `Log.CloseAndFlushAsync()` atomically replaces `Log.Logger` with `Logger.None`, the silent logger. 2. It awaits `DisposeAsync()` on the old logger (or calls `Dispose()` if only that is available). 3. Any later `Log.X(...)` call is silently discarded. The synchronous `Log.CloseAndFlush()` does the same with `Dispose()`. A logger you created yourself and did not assign to `Log.Logger` must be disposed directly. ## The Program.cs pattern ```csharp Log.Logger = new LoggerConfiguration() .WriteTo.Console() .WriteTo.File("logs/callcentre-.txt", rollingInterval: RollingInterval.Day, buffered: true) .CreateLogger(); try { Log.Information("Call centre starting"); await RunCallCentreAsync(); } catch (Exception ex) { Log.Fatal(ex, "Call centre terminated unexpectedly"); } finally { await Log.CloseAndFlushAsync(); } ``` The `catch` records the crash as a `Fatal` event; the `finally` makes sure the buffers containing it are written before `Main` returns. ## Tuning BatchingOptions for the workload The defaults suit a long-running service. Other shapes call for changes: - **Short-lived jobs** (a nightly call-recording export, say) may finish before a two-second window closes; lowering `BufferingTimeLimit` shortens the at-risk window, and disposal still drains the rest. - **High-volume services** can keep `BatchSizeLimit` at 1000 but should think about `QueueLimit`: setting it to `null` makes the queue unbounded, trading dropped events for memory growth during an outage. - **Outage behaviour**: the source documents lowering `RetryTimeLimit` to reduce buffering under high load, so a dead destination costs less memory and fewer stale events. Whatever the values, the first rule stands: the logger must be disposed on the way out, or the last batch is never sent. ## What flushing cannot fix - A process that is killed outright (an out-of-memory kill, or a container stopped once its grace period runs out) never reaches `finally`; anything in memory is lost. Keep buffering windows short for that reason. - If a batched sink's destination is unreachable, the batching sink keeps retrying for up to `RetryTimeLimit` and the queue can reach `QueueLimit`, after which new events are dropped. Disposal can only flush to a destination that accepts the batch. - Disposing early is its own bug: code that logs after `CloseAndFlushAsync()` (a hosted service still stopping, say) writes into `Logger.None`.

  • Does Log.CloseAndFlushAsync() help with a logger you created and injected but never assigned to Log.Logger?
    No. It only disposes the logger currently stored in `Log.Logger`. A separately created `Logger` must be disposed by whoever owns it, for example with `await using` or by registering it so the container disposes it on shutdown.
  • With default BatchingOptions, why does the first event of a run appear immediately while later ones arrive in groups?
    `EagerlyEmitFirstEvent` defaults to `true`, so the batching sink sends the first event straight away for perceived liveness. After that it waits until 1000 events are queued or two seconds pass, whichever comes first.
  • What happens to events when a batched sink's destination is down for a long time?
    Failed batches are retried for up to `RetryTimeLimit` (10 minutes by default) while new events keep queuing. Once the queue holds `QueueLimit` events (100000 by default), further events are discarded, and the failures are reported through the failure listener, which writes to `SelfLog` unless replaced.

A batched sink is a mail room that collects letters and sends a van every two seconds or when a thousand pile up. Closing the office without calling the last van leaves the final letters on the table; CloseAndFlushAsync is the closing routine that waits for that last van to leave.

  • Serilog writes every event synchronously, so nothing can be lost on exit
  • A batched sink only sends once 1000 events have accumulated
  • After CloseAndFlushAsync, Log.Logger keeps logging to the same sinks
  • CloseAndFlushAsync belongs at the top of Main, before the app runs
  • A finally block with CloseAndFlush also saves logs when the process is killed

Assumes Serilog 4.4, where IBatchedLogEventSink, BatchingOptions and Log.CloseAndFlushAsync() are part of the core Serilog package.