skip to content

What is plugin.path in Kafka Connect, and how does the runtime isolate connector plugins? What problems does misconfiguring it cause?

level: seniorimportance: should knowfreq 50%

answer

  1. plugin.path = dirs of plugins
  2. one classloader per plugin
  3. point at parent dir, not the JARs
  4. converters & SMTs are plugins too
  5. isolation avoids dep clashes

basics

~10 s

plugin.path is a list of directories where the worker finds connector/converter/transform JARs. Connect loads each plugin in an isolated classloader to avoid dependency conflicts. Misconfiguring it causes 'class not found' errors or dependency clashes.

solid answer

~40 s

`plugin.path` is a worker property listing filesystem directories that contain Connect plugins (connectors, converters, SMTs, predicates), each typically packaged as its own subdirectory of JARs (an 'uber-jar' or a directory of dependencies). Connect uses **classloading isolation**: each plugin gets a dedicated `PluginClassLoader` so one connector's dependency (say a particular Jackson or AWS SDK version) can't conflict with another's or with Connect's own. The plugins must sit in subdirectories of a `plugin.path` entry, not on the main `CLASSPATH`. Common misconfigurations: pointing at the directory *of* JARs rather than its parent, leaving plugins on the classpath (defeating isolation and causing version clashes), or forgetting converters/transforms are also plugins. Symptoms are `ClassNotFoundException`/connector class not found at `POST /connectors`, or `NoSuchMethodError`/`LinkageError` from leaked dependencies.

go deeper

for a junior

Know plugin.path tells the worker where to find connector JARs.

for a middle

Explain the directory-of-subdirectories layout and that converters/SMTs count as plugins.

for a senior

Explain per-plugin classloader isolation and the dependency-conflict problems it prevents/causes.

for a principal

Define the org's plugin packaging/deployment standard (container mounts, isolation, version governance).

## plugin.path and classloader isolation ### What a 'plugin' is In Connect, a **plugin** is any pluggable component the runtime loads: connectors (source/sink), **converters** (JSON/Avro/Protobuf serializers), **transforms** (Single Message Transforms, SMTs), and **predicates**. Each is packaged as a set of JARs. ### What `plugin.path` is `plugin.path` is a comma-separated list of **directories** in the worker config, e.g. `plugin.path=/usr/share/java,/opt/connectors`. Inside each listed directory, every **immediate subdirectory** (or uber-jar) is treated as one plugin and gets its own classloader. So the layout is: ``` /opt/connectors/ <- listed in plugin.path debezium-postgres/ <- one plugin *.jar s3-sink/ <- another plugin *.jar ``` ### Classloader isolation (why it exists) Different connectors bundle conflicting transitive dependencies (different versions of Jackson, Netty, guava, AWS/GCP SDKs). If they all loaded into one flat classpath, you'd get version clashes and runtime `NoSuchMethodError`/`LinkageError`. Connect solves this with a **`PluginClassLoader` per plugin**: each plugin's classes and dependencies are isolated; only a small set of shared API/framework classes come from the parent loader. This lets two connectors use incompatible library versions side by side. ### Misconfiguration symptoms - **Pointing `plugin.path` at the JAR directory itself** instead of its parent: Connect scans subdirectories, so the plugin isn't discovered → 'Failed to find any class that implements Connector' / `ClassNotFoundException` when you `POST /connectors`. - **Putting plugins on the main `CLASSPATH`** (the old pre-isolation approach): they load in the shared loader, breaking isolation and causing dependency clashes between connectors. - **Forgetting converters/SMTs are plugins**: if your Avro converter lives only on the classpath but the connector is isolated (or vice versa), you can hit class-visibility errors. - **Multiple copies of the same plugin** across `plugin.path` entries → ambiguous loading / version confusion. ### Operational notes - At startup the worker logs the plugins it discovered; checking that log is the fastest way to confirm a plugin is on `plugin.path`. - Converters referenced in `key.converter`/`value.converter` must be discoverable as plugins too. - In containers, mount connector JARs into a `plugin.path` directory rather than baking them onto the classpath.

  • Why does Connect use a separate classloader per plugin instead of one shared classpath?
    To isolate each plugin's transitive dependencies, so connectors bundling conflicting library versions (e.g., different Jackson/AWS SDK versions) can coexist without LinkageError/NoSuchMethodError clashes.
  • You POST a connector and get 'Failed to find any class that implements Connector'. What's the likely cause?
    The plugin isn't on plugin.path — often plugin.path points directly at the JAR folder instead of its parent directory, so Connect's subdirectory scan never finds the plugin. Fix the path and restart the worker.

  • Saying you should add connector JARs to the main CLASSPATH (that defeats isolation).
  • Treating plugin.path as a single JAR file rather than directories of plugin subdirectories.
  • Forgetting that converters and SMTs are also plugins loaded from plugin.path.

context