skip to content

Compare Boot 3.4 native structured logging with the encoder-based approach (logstash-logback-encoder). How would you architect JSON logging for an aggregation pipeline across many services?

level: principalimportance: nice to knowfreq 22%

answer

  1. native = 1 property, no dep, no XML; encoder = dependency + logback-spring.xml
  2. encoder more configurable but Logback-coupled + more to own
  3. stdout NDJSON -> collector (Fluent Bit/Vector/OTel), not app-shipped
  4. shared starter enforces one schema + service.* + Micrometer traceId
  5. field names are a contract; JSON size + redaction cost

basics

~20 s

Boot 3.4 native structured logging gives JSON via one property with no XML or extra dependency, covering ecs/logstash/gelf. The encoder approach (logstash-logback-encoder in logback-spring.xml) is more configurable but heavier. Prefer native; drop to the encoder only for needs native can't express.

solid answer

~40 s

**Native (Boot 3.4):** enable with `logging.structured.format.console/file=ecs|logstash|gelf`, no dependency, no `logback-spring.xml`; extend via `logging.structured.json.*`, `service.*` properties, `StructuredLoggingJsonMembersCustomizer`, or a `StructuredLogFormatter`. It's uniform across Logback/Log4j2 and configured through the same property system as everything else. **Encoder-based:** add `net.logstash.logback:logstash-logback-encoder`, declare `<encoder class="...LogstashEncoder">` in `logback-spring.xml`, giving very fine-grained control (custom providers, nested objects, per-appender shaping) at the cost of XML, a dependency, and Logback coupling. **Architecture:** standardize on native ECS emitted to **stdout** (twelve-factor), let the platform collector (Fluent Bit/Vector/OTel Collector) ship to the sink; centralize the field taxonomy (service.name/version/env, traceId/spanId from Micrometer Tracing) via a shared starter so all services share one schema; keep dashboards/alerts as the field contract. Use the encoder only where native genuinely falls short.

code

java · 32 lines
java
// NATIVE (Boot 3.4) -- application.properties, no dependency, no XML:
//   logging.structured.format.console=ecs
//   logging.structured.ecs.service.name=${spring.application.name}
//   logging.structured.ecs.service.environment=prod
// Ship stdout via the platform collector (Fluent Bit / Vector / OTel Collector).

// ENCODER-BASED (pre-3.4 style) -- build.gradle:
//   implementation 'net.logstash.logback:logstash-logback-encoder:7.4'
// logback-spring.xml:
//   <appender name="JSON" class="ch.qos.logback.core.ConsoleAppender">
//     <encoder class="net.logstash.logback.encoder.LogstashEncoder"/>
//   </appender>

// Central schema/redaction with native, applied to ALL services via a shared starter:
// META-INF/spring.factories registers this (NOT a bean -- early logging init):
//   org.springframework.boot.logging.structured.StructuredLoggingJsonMembersCustomizer=\
//     com.example.logging.RedactAndTagCustomizer
package com.example.logging;

import ch.qos.logback.classic.spi.ILoggingEvent;
import org.springframework.boot.json.JsonWriter;
import org.springframework.boot.logging.structured.StructuredLoggingJsonMembersCustomizer;

public class RedactAndTagCustomizer
        implements StructuredLoggingJsonMembersCustomizer<ILoggingEvent> {
    @Override
    public void customize(JsonWriter.Members<ILoggingEvent> members) {
        members.add("schema.version", e -> "1");   // fleet-wide field
        members.applyingNameProcessor((name) -> name); // hook point for renames
        // members.remove("password") / redaction logic centralized here
    }
}

go deeper

for a junior

Know native gives JSON with one property and the old way used an encoder + XML.

for a middle

Contrast dependency/XML/coupling and know native has extension points.

for a senior

Argue native-by-default, stdout+collector, Micrometer correlation, and when the encoder still wins.

for a principal

Design fleet-wide: shared starter, versioned schema contract, source-format vs collector-normalize, cost/redaction/reliability tradeoffs.

This is a build-vs-adopt and standardization question layered on Spring Boot 3.4's logging capabilities. **The two mechanisms.** *Native structured logging (Boot 3.4+):* - **Enable:** `logging.structured.format.console=ecs` / `logging.structured.format.file=...` (also `logstash`, `gelf`, or a custom `StructuredLogFormatter` FQN). - **No dependency, no XML** for the common path; it hooks into whichever logging framework Boot uses (Logback default, or Log4j2). - **Extension points:** `logging.structured.json.add/rename/exclude/include` (static fields), `logging.structured.<format>.service.*` (service identity), `StructuredLoggingJsonMembersCustomizer` (programmatic members, `spring.factories`-registered), full `StructuredLogFormatter<E>` (whole new shape). - **Pros:** minimal config, unified with Boot's property system and profiles, framework-portable, less to maintain, evolves with Boot. - **Cons:** newer, fewer knobs than the mature encoder; deeply custom serialization needs code. *Encoder-based (pre-3.4 idiom, still valid):* - Add `net.logstash.logback:logstash-logback-encoder`; configure a `logback-spring.xml` with `<encoder class="net.logstash.logback.encoder.LogstashEncoder">` (or `LoggingEventCompositeJsonEncoder` with a stack of providers). - **Pros:** battle-tested, extremely configurable — composite providers, arbitrary nested JSON, per-appender custom shaping, async appenders, direct-to-TCP/UDP. - **Cons:** extra dependency, XML config, tied to Logback, and now duplicates what Boot offers natively; more surface to own. **When to drop to the encoder:** you need a serialization shape or provider that native + `StructuredLogFormatter` can't reach ergonomically, you're on a large legacy `logback-spring.xml` you don't want to rewrite, or you need encoder-only features (e.g. certain composite providers / direct network appenders). Otherwise native is the lower-maintenance default. **Architecting for a multi-service pipeline:** 1. **Emit to stdout, not to files or sockets from the app.** Twelve-factor: the app writes newline-delimited JSON to stdout; the platform (Kubernetes/Docker/Railway) captures it. A **sidecar/agent collector** — Fluent Bit, Fluentd, Vector, or the **OpenTelemetry Collector** — ships and (if needed) transforms it. Keep transport out of the app so you don't couple business services to the log backend's availability. 2. **Pick one schema and enforce it fleet-wide.** Choose the format that matches the sink (ECS for Elastic). Encapsulate the choice and the shared fields in an **internal Boot starter / shared config** so every service inherits identical `service.*`, added fields, and renames. A consistent schema is what makes cross-service queries and dashboards possible. 3. **Wire correlation in.** Enable **Micrometer Tracing** so `traceId`/`spanId` land in MDC and thus in every JSON line — this is what links logs to distributed traces and to metrics exemplars. Standardize a small set of MDC keys (tenant, request id) via a filter. 4. **Treat field names as a contract.** Dashboards, alerts, and saved queries key on field names; renames/exclusions are breaking changes. Version the schema and roll changes through the shared starter, not per-service. 5. **Cost and reliability.** Structured JSON is larger than plain lines — budget ingest/storage; consider sampling debug logs, and never block the request thread on log shipping (async appender or, preferably, collector-side buffering). Guard against sensitive data in logged fields (a JSON member customizer can redact centrally). 6. **Decouple format at source vs normalize at collector.** If you can't standardize the source format across polyglot services, normalize in the collector (OTel/Vector) instead — but for a Boot fleet, standardizing at source via a shared starter is simpler and cheaper than collector-side remapping. **Summary judgment:** default to **native ECS to stdout + shared starter + Micrometer correlation + platform collector**; reserve the logstash-logback-encoder for the residual cases native can't express. This keeps services thin, schema uniform, and the pipeline replaceable.

  • Why emit JSON to stdout and let a collector ship it, rather than shipping from the app?
    Twelve-factor separation: the app treats logs as an event stream to stdout and stays unaware of the backend. A platform collector (Fluent Bit/Vector/OTel) handles buffering, retries, and transport, so a sink outage or format change doesn't affect request threads or require redeploying services.
  • When is the logstash-logback-encoder still the better choice over native structured logging?
    When you need encoder-only features (rich composite providers, direct TCP/UDP appenders, or a serialization shape awkward to express with StructuredLogFormatter), or you're maintaining a large existing logback-spring.xml you don't want to rewrite.
  • How do you keep the JSON schema consistent across dozens of services?
    Package the format choice, `service.*` metadata, added fields, and any `StructuredLoggingJsonMembersCustomizer` in a shared internal Boot starter so every service inherits the same schema; treat field names as a versioned contract with the dashboards.

saying these in an interview costs you the question

  • Claiming Boot 3.4 native structured logging still requires the logstash-logback-encoder dependency
  • Coupling each service to the log backend by shipping logs directly instead of via a collector
  • Letting each service define its own ad-hoc JSON schema (no shared starter)
  • Ignoring that JSON logs are larger and can block threads if shipped synchronously
  • Renaming/removing fields without treating them as a dashboard contract

context

resolving links…