skip to content

How do you create a new connector via the Kafka Connect REST API, and what is the minimal request body?

level: juniorimportance: must knowfreq 75%

answer

  1. POST /connectors
  2. body = name + config (flat strings)
  3. connector.class required
  4. 201 created, 409 if name exists
  5. PUT /config = idempotent upsert

basics

~10 s

Send a POST to /connectors with a JSON body containing a top-level "name" and a "config" object (which must repeat connector.class and the connector's settings). Connect returns 201 Created with the connector info.

solid answer

~40 s

Kafka Connect runs an embedded HTTP server (default port 8083). To create a connector you POST JSON to /connectors. The body has two top-level keys: name (the connector's unique name) and config (a flat map of string key/value settings). The config must include connector.class, tasks.max, and any connector-specific properties (topics, connection URLs, converters, etc.). On success you get 201 Created with {name, config, tasks, type}. The name must be unique; re-POSTing an existing name returns 409 Conflict. An alternative idempotent path is PUT /connectors/{name}/config with just the flat config map (no wrapping name/config), which creates the connector if absent or updates it if present.

go deeper

for a junior

Know the verb+path (POST /connectors), the two-field body, and that connector.class is required.

for a middle

Distinguish POST vs PUT /config semantics, status codes 201/409/400, and that config values are strings.

for a senior

Explain validation-before-accept, cluster coordination across workers, and why PUT /config is preferred for declarative tooling.

for a principal

Frame the REST API as the control plane for a distributed cluster; reason about idempotency, GitOps reconciliation, and forwarding to the leader.

**Kafka Connect** is a framework for streaming data between Kafka and external systems using reusable **connectors** (source connectors pull data into Kafka; sink connectors push data out). Connect runs as a cluster of **worker** JVM processes; in **distributed mode** every worker exposes the same REST API (default `http://localhost:8083`) and the cluster coordinates config/offset/status through internal Kafka topics. **Creating a connector — POST /connectors.** The request body is JSON with exactly two top-level fields: ``` { "name": "my-source", "config": { "connector.class": "io.confluent.connect.jdbc.JdbcSourceConnector", "tasks.max": "1", "connection.url": "jdbc:postgresql://...", "topic.prefix": "db-" } } ``` Key points: - **`name`** identifies the connector cluster-wide and must be unique. POSTing a name that already exists returns **409 Conflict**. - **`config`** is a flat map of **string-to-string** values — even numbers like `tasks.max` are JSON strings. - **`connector.class`** is mandatory and selects the connector plugin (must be on the worker's plugin path). - A successful create returns **201 Created** with the connector's current `{name, config, tasks, type}`. **Alternative: PUT /connectors/{name}/config.** This endpoint is idempotent and upsert-style: the body is just the flat config map (no `name`/`config` wrapper). If the connector doesn't exist it is created (201); if it does it is reconfigured (200). This is the preferred form for declarative tooling (GitOps, scripts) because re-applying the same config is safe and doesn't 409. **Edge cases:** Connect validates the config before accepting it; a config that fails validation returns **400 Bad Request** with the per-field error messages (the same structure returned by PUT /connector-plugins/{class}/config/validate). The REST request is handled by whichever worker received it, which then forwards/coordinates with the cluster leader, so the create is durable across the whole Connect cluster, not just one worker.

  • What happens if you POST the same connector name twice?
    The second POST returns 409 Conflict because connector names must be unique. To change an existing connector use PUT /connectors/{name}/config instead, which updates in place.
  • What is the difference in body shape between POST /connectors and PUT /connectors/{name}/config?
    POST /connectors wraps the settings: {name, config:{...}}. PUT /connectors/{name}/config takes only the flat config map directly (no name or config wrapper), and the name comes from the URL path.

saying these in an interview costs you the question

  • Claiming the config can be a nested object — config is a flat string-to-string map.
  • Saying POST is idempotent — POST 409s on duplicate name; PUT /config is the idempotent path.
  • Forgetting connector.class is mandatory.
  • Thinking Connect runs on port 9092 (that's the broker; Connect's REST API is 8083).

context

resolving links…