skip to content

What is plugin.path in Kafka Connect, and how does the runtime isolate connector plugins? What problems does misconfiguring it cause?

level: seniorimportance: should knowfreq 50%

answer

  1. plugin.path = dirs of plugins
  2. one classloader per plugin
  3. point at parent dir, not the JARs
  4. converters & SMTs are plugins too
  5. isolation avoids dep clashes

basics

~10 s

plugin.path is a list of directories where the worker finds connector/converter/transform JARs. Connect loads each plugin in an isolated classloader to avoid dependency conflicts. Misconfiguring it causes 'class not found' errors or dependency clashes.

solid answer

~40 s

`plugin.path` is a worker property listing filesystem directories that contain Connect plugins (connectors, converters, SMTs, predicates), each typically packaged as its own subdirectory of JARs (an 'uber-jar' or a directory of dependencies). Connect uses **classloading isolation**: each plugin gets a dedicated `PluginClassLoader` so one connector's dependency (say a particular Jackson or AWS SDK version) can't conflict with another's or with Connect's own. The plugins must sit in subdirectories of a `plugin.path` entry, not on the main `CLASSPATH`. Common misconfigurations: pointing at the directory *of* JARs rather than its parent, leaving plugins on the classpath (defeating isolation and causing version clashes), or forgetting converters/transforms are also plugins. Symptoms are `ClassNotFoundException`/connector class not found at `POST /connectors`, or `NoSuchMethodError`/`LinkageError` from leaked dependencies.

go deeper

for a junior

Know plugin.path tells the worker where to find connector JARs.

for a middle

Explain the directory-of-subdirectories layout and that converters/SMTs count as plugins.

for a senior

Explain per-plugin classloader isolation and the dependency-conflict problems it prevents/causes.

for a principal

Define the org's plugin packaging/deployment standard (container mounts, isolation, version governance).

## plugin.path and classloader isolation ### What a 'plugin' is In Connect, a **plugin** is any pluggable component the runtime loads: connectors (source/sink), **converters** (JSON/Avro/Protobuf serializers), **transforms** (Single Message Transforms, SMTs), and **predicates**. Each is packaged as a set of JARs. ### What `plugin.path` is `plugin.path` is a comma-separated list of **directories** in the worker config, e.g. `plugin.path=/usr/share/java,/opt/connectors`. Inside each listed directory, every **immediate subdirectory** (or uber-jar) is treated as one plugin and gets its own classloader. So the layout is: ``` /opt/connectors/ <- listed in plugin.path debezium-postgres/ <- one plugin *.jar s3-sink/ <- another plugin *.jar ``` ### Classloader isolation (why it exists) Different connectors bundle conflicting transitive dependencies (different versions of Jackson, Netty, guava, AWS/GCP SDKs). If they all loaded into one flat classpath, you'd get version clashes and runtime `NoSuchMethodError`/`LinkageError`. Connect solves this with a **`PluginClassLoader` per plugin**: each plugin's classes and dependencies are isolated; only a small set of shared API/framework classes come from the parent loader. This lets two connectors use incompatible library versions side by side. ### Misconfiguration symptoms - **Pointing `plugin.path` at the JAR directory itself** instead of its parent: Connect scans subdirectories, so the plugin isn't discovered → 'Failed to find any class that implements Connector' / `ClassNotFoundException` when you `POST /connectors`. - **Putting plugins on the main `CLASSPATH`** (the old pre-isolation approach): they load in the shared loader, breaking isolation and causing dependency clashes between connectors. - **Forgetting converters/SMTs are plugins**: if your Avro converter lives only on the classpath but the connector is isolated (or vice versa), you can hit class-visibility errors. - **Multiple copies of the same plugin** across `plugin.path` entries → ambiguous loading / version confusion. ### Operational notes - At startup the worker logs the plugins it discovered; checking that log is the fastest way to confirm a plugin is on `plugin.path`. - Converters referenced in `key.converter`/`value.converter` must be discoverable as plugins too. - In containers, mount connector JARs into a `plugin.path` directory rather than baking them onto the classpath.

  • Why does Connect use a separate classloader per plugin instead of one shared classpath?
    To isolate each plugin's transitive dependencies, so connectors bundling conflicting library versions (e.g., different Jackson/AWS SDK versions) can coexist without LinkageError/NoSuchMethodError clashes.
  • You POST a connector and get 'Failed to find any class that implements Connector'. What's the likely cause?
    The plugin isn't on plugin.path — often plugin.path points directly at the JAR folder instead of its parent directory, so Connect's subdirectory scan never finds the plugin. Fix the path and restart the worker.

saying these in an interview costs you the question

  • Saying you should add connector JARs to the main CLASSPATH (that defeats isolation).
  • Treating plugin.path as a single JAR file rather than directories of plugin subdirectories.
  • Forgetting that converters and SMTs are also plugins loaded from plugin.path.

context

resolving links…