skip to content

Connect Security and Config Externalization

3 roadmaps5 questions

Securing Connect: per-connector broker credentials and override policy, externalized secrets, and a protected REST endpoint. Interviewers ask because connector configs are a classic place for plaintext passwords.

part ofApache Kafkaoverview, primer and where to startread it →
on this pageshow

explore

questions

5

How do you configure a Kafka Connect worker to connect to brokers secured with TLS and SASL, and which prefixes apply to which client?

level: juniorimportance: must knowfreq 70%

answer

  1. security.protocol=SASL_SSL at top level
  2. producer. / consumer. / admin. prefixes
  3. sasl.jaas.config is inline, not a file
  4. top-level = all clients incl. worker internal
  5. truststore trusts broker; keystore = mTLS identity

basics

~20 s

Put the broker security settings (security.protocol, SASL/SSL configs) in the worker properties file. By default they apply to all of Connect's internal clients. Prefix with producer., consumer., or admin. to target a specific client type.

solid answer

~40 s

Kafka Connect runs several internal Kafka clients: a producer (sink offsets / source records), consumer (sink connectors), and admin (creating internal topics). To reach a secured cluster you set top-level keys in the worker config such as security.protocol=SASL_SSL, sasl.mechanism=SCRAM-SHA-512, sasl.jaas.config, and the ssl.truststore.location/password. These top-level keys configure all Connect clients including the worker's own group-membership and internal-topic clients. To override settings for just one client you prefix them: producer.<config>, consumer.<config>, admin.<config>. For source connectors the producer prefix matters; for sink connectors the consumer prefix. The same security keys also need the bootstrap.servers pointed at the secured listener. JAAS is supplied inline via sasl.jaas.config rather than a separate file.

go deeper

for a junior

Know that security settings go in the worker properties, and that there are producer./consumer./admin. prefixes.

for a middle

Explain which client each prefix targets and that top-level keys cover the worker's internal clients.

for a senior

Reason about per-client principals, the internal-topic client gotcha, and SASL_SSL vs mTLS trade-offs.

for a principal

Design a credential/principal strategy across source/sink/admin clients and the worker, with rotation and least privilege.

**What Kafka Connect is:** Connect is a framework for streaming data between Kafka and external systems using reusable plugins called connectors. A *source* connector reads from an external system and writes to Kafka; a *sink* connector reads from Kafka and writes out. Connectors run inside JVM processes called *workers*. **Why there are multiple clients:** A worker doesn't talk to Kafka as a single client. It opens: (1) a **producer** to write source-connector records and to write offsets/status/config to internal topics; (2) a **consumer** for sink connectors to read their input topics; (3) an **admin** client to create internal topics and inspect the cluster; and (4) the distributed worker's own group-coordination client. When the broker is secured, every one of these must present valid credentials. **TLS (SSL) settings** encrypt the connection and let the client verify the broker's certificate. Key configs: `security.protocol` (one of PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL), `ssl.truststore.location` and `ssl.truststore.password` (to trust the broker cert), and for mutual TLS `ssl.keystore.location`/`ssl.keystore.password`/`ssl.key.password`. **SASL settings** authenticate the client to the broker. Key configs: `sasl.mechanism` (e.g. PLAIN, SCRAM-SHA-256/512, GSSAPI for Kerberos, OAUTHBEARER) and `sasl.jaas.config`, an inline JAAS string like `org.apache.kafka.common.security.scram.ScramLoginModule required username="connect" password="...";`. **Where they go and prefixing rules:** Top-level keys in the worker `.properties` (or env vars in container images) apply to *all* Connect clients. To target a specific client, prefix the key: - `producer.security.protocol`, `producer.sasl.jaas.config`, ... → the producer. - `consumer.<key>` → sink consumer. - `admin.<key>` → admin client. This matters when, for instance, source and sink connectors need different identities, or you want a dedicated principal for the admin client that creates internal topics. **Edge cases:** the worker's own internal client (for the offset/config/status topics in distributed mode) follows the *top-level* settings, not the producer/consumer prefixes; if you only set `producer.*` you'll authenticate source records but the worker itself may fail to read its config topic. Always set the top-level security block too.

  • If you set only producer.* security configs, why might the distributed worker still fail to start?
    The worker's own internal client that reads/writes the config, offset, and status topics uses the top-level security settings, not the producer. prefix. Without top-level configs it can't authenticate to those internal topics.
  • How do you supply SASL credentials — JAAS file or inline?
    Kafka clients support inline sasl.jaas.config, which is the idiomatic approach for Connect. You don't need a separate java.security.auth.login.config file, though it still works via the JVM system property.

saying these in an interview costs you the question

  • Saying you must use a separate JAAS file (inline sasl.jaas.config is standard)
  • Claiming producer.* settings cover the worker's internal config/offset/status topic clients
  • Confusing truststore (trust broker cert) with keystore (your own mTLS identity)
  • Thinking PLAINTEXT vs SSL is the same axis as SASL (security.protocol combines both)

context

resolving links…

open as a page

How does ConfigProvider (e.g. FileConfigProvider) let you keep secrets out of connector configs, and how does the ${...} indirection work?

level: middleimportance: must knowfreq 65%

basics

~10 s

A ConfigProvider resolves ${provider:[path:]key} placeholders at runtime so secrets aren't stored literally in connector configs. You register a provider in the worker config (e.g. config.providers=file with FileConfigProvider) and reference values like ${file:/secrets.properties:db.password}.

open as a page

What does connector.client.config.override.policy do, and why is it set to None by default?

level: middleimportance: must knowfreq 60%

basics

~10 s

It's a worker-level setting that controls whether individual connectors can override the worker's Kafka client configs using producer.override./consumer.override./admin.override. prefixes. Defaults to None (no overrides allowed); common values are All or Principal.

open as a page

How do you secure the Kafka Connect REST API with TLS and authentication, and what's the listeners.https configuration?

level: seniorimportance: should knowfreq 45%

basics

~10 s

Set listeners=https://host:port plus listeners.https.ssl.* keystore/truststore configs for TLS. For auth, plug in a REST extension via rest.extension.classes (e.g. BasicAuthSecurityRestExtension for HTTP Basic) since Connect has no built-in user store.

open as a page

Design how a multi-tenant Connect cluster gives each connector its own broker identity for per-tenant ACLs. What pieces fit together?

level: principalimportance: should knowfreq 35%

basics

~10 s

Set connector.client.config.override.policy=Principal on the worker, then give each connector its own credentials via producer.override./consumer.override.sasl.jaas.config (resolved from a ConfigProvider). Define per-principal broker ACLs so each tenant can only touch its own topics.

open as a page