skip to content

Why is mergeServiceFiles() needed in a Shadow build, and what breaks if you omit it?

level: middleimportance: must knowfreq 60%

answer

  1. META-INF/services collisions
  2. ServiceLoader providers dropped
  3. concatenate not overwrite
  4. JDBC driver / SLF4J binding missing
  5. transformer

basics

~10 s

Multiple dependencies can each ship a file at the same path under META-INF/services/. Without mergeServiceFiles() one overwrites the others, so some ServiceLoader providers vanish and features silently break.

solid answer

~40 s

Java's `ServiceLoader` SPI discovers implementations via text files named after the service interface under `META-INF/services/` (each lists provider class names). Many libraries ship such files — and several jars can contain a file at the **exact same path**, e.g. `META-INF/services/java.sql.Driver`. When Shadow flattens everything into one jar, two entries at the same path collide; the default copy strategy keeps only one, so providers from the losing jars disappear. `mergeServiceFiles()` is a Shadow **transformer** that detects these `META-INF/services/*` collisions and **concatenates** their provider lists instead of overwriting. Symptoms of omitting it: `ServiceLoader` finds fewer providers than expected — missing JDBC drivers, missing SLF4J bindings, missing Jackson modules, Reactor/Netty providers, etc. The fix is one line inside the `shadowJar` block. Related transformers handle other mergeable formats (e.g. `Log4j2PluginsCacheFileTransformer`, properties, Netty's `META-INF/native-image`).

code

kotlin · 5 lines
kotlin
tasks.shadowJar {
    mergeServiceFiles()              // concat META-INF/services/*
    append("META-INF/spring.handlers")
    append("META-INF/spring.schemas")
}

go deeper

for a junior

Know that mergeServiceFiles() combines META-INF/services files so providers aren't lost.

for a middle

Explain ServiceLoader/SPI, the same-path collision, concat vs overwrite, and concrete symptoms like missing JDBC drivers.

for a senior

Discuss companion transformers (Spring handlers, Log4j2 cache, Groovy extensions) and how to diagnose by unzipping the jar.

for a principal

Treat resource-merge policy as part of a packaging standard, ensuring SPI/plugin registries survive shading across all services.

## The Java Service Provider Interface Java's `java.util.ServiceLoader` lets a library expose an interface and let *consumers* drop in implementations discovered at runtime. The wiring is a plain text file: ``` META-INF/services/<fully.qualified.ServiceInterface> ``` whose lines are the fully-qualified names of provider classes. `ServiceLoader.load(Foo::class.java)` reads **all** such files visible on the classpath and instantiates the listed providers. Crucially, multiple jars may each ship a file at the *same path* — e.g. three JDBC drivers all provide `META-INF/services/java.sql.Driver`. ## Why a fat jar breaks this A jar (zip) cannot have two entries at the identical path. When Shadow merges jar A and jar B that both contain `META-INF/services/java.sql.Driver`, only one survives. The default behavior is **last-write-wins / first-wins**, silently dropping the other providers. Nothing errors at build time; you discover it at runtime when `ServiceLoader` only finds one driver, or your logging backend isn't picked up. ## The fix: a transformer Shadow's **transformers** post-process entries during the merge. `mergeServiceFiles()` registers a `ServiceFileTransformer` that, for any path under `META-INF/services/`, **concatenates** the line contents of all colliding files into one merged file. ```kotlin tasks.named<com.github.jengelman.gradle.plugins.shadow.tasks.ShadowJar>("shadowJar") { mergeServiceFiles() } ``` or in the modern API simply inside the `shadowJar { }` accessor: ```kotlin shadowJar { mergeServiceFiles() } ``` You can scope it: `mergeServiceFiles("META-INF/services")` or use the configuring overload to include/exclude specific service files. ## Other merge transformers The same collision class affects other resource formats. Common companions: - `mergeGroovyExtensionModules()` — Groovy extension descriptors. - `append("META-INF/spring.handlers")` / `append("META-INF/spring.schemas")` — Spring's namespace handler files. - `Log4j2PluginsCacheFileTransformer` — Log4j2's binary plugin cache. - `transform(...)` with a custom `Transformer` for bespoke formats. ## Mental model Think of `mergeServiceFiles()` as turning an overwrite into an append for SPI registries. Any time discovery is registry-file-based and several jars register, you need a merge transformer or the registry ends up truncated. ## Diagnosing Unzip the produced jar and inspect `META-INF/services/` — if a file you expect to list several providers lists only one, your transformer is missing or mis-scoped.

  • Give a concrete runtime symptom of a missing mergeServiceFiles().
    A JDBC `java.sql.Driver` provider isn't found (`No suitable driver`), or an SLF4J/logging binding or Jackson module silently isn't registered, because only one provider file survived the merge.
  • How would you handle Spring's spring.handlers/spring.schemas in a fat jar?
    Those are line/property files that must be concatenated too — use `append("META-INF/spring.handlers")` and `append("META-INF/spring.schemas")` (Spring Boot's bootJar handles this automatically, but a plain Shadow build does not).
  • Does mergeServiceFiles() solve duplicate .class file collisions?
    No — it only merges service descriptor text files. Class-file duplicates from overlapping dependencies are a separate concern handled by exclusions, relocation, or accepting first-wins.

Two guest lists for the same event get merged into one combined list — without merging, only the last list shown at the door gets in and everyone on the others is turned away.

saying these in an interview costs you the question

  • Saying ServiceLoader files are automatically merged by Shadow without the transformer.
  • Confusing service-file merging with class duplicate handling.
  • Claiming the failure is a build-time error — it's a silent runtime defect.

context

resolving links…