skip to content

Connect REST API and Lifecycle Management

3 roadmaps6 questions

Driving connectors through the REST API to create, pause, restart, and inspect the status of a stuck task. Practical operations knowledge that shows you have run Connect rather than only configured it.

part ofApache Kafkaoverview, primer and where to startread it →
on this pageshow

explore

questions

6

How do you create a new connector via the Kafka Connect REST API, and what is the minimal request body?

level: juniorimportance: must knowfreq 75%

answer

  1. POST /connectors
  2. body = name + config (flat strings)
  3. connector.class required
  4. 201 created, 409 if name exists
  5. PUT /config = idempotent upsert

basics

~10 s

Send a POST to /connectors with a JSON body containing a top-level "name" and a "config" object (which must repeat connector.class and the connector's settings). Connect returns 201 Created with the connector info.

solid answer

~40 s

Kafka Connect runs an embedded HTTP server (default port 8083). To create a connector you POST JSON to /connectors. The body has two top-level keys: name (the connector's unique name) and config (a flat map of string key/value settings). The config must include connector.class, tasks.max, and any connector-specific properties (topics, connection URLs, converters, etc.). On success you get 201 Created with {name, config, tasks, type}. The name must be unique; re-POSTing an existing name returns 409 Conflict. An alternative idempotent path is PUT /connectors/{name}/config with just the flat config map (no wrapping name/config), which creates the connector if absent or updates it if present.

go deeper

for a junior

Know the verb+path (POST /connectors), the two-field body, and that connector.class is required.

for a middle

Distinguish POST vs PUT /config semantics, status codes 201/409/400, and that config values are strings.

for a senior

Explain validation-before-accept, cluster coordination across workers, and why PUT /config is preferred for declarative tooling.

for a principal

Frame the REST API as the control plane for a distributed cluster; reason about idempotency, GitOps reconciliation, and forwarding to the leader.

**Kafka Connect** is a framework for streaming data between Kafka and external systems using reusable **connectors** (source connectors pull data into Kafka; sink connectors push data out). Connect runs as a cluster of **worker** JVM processes; in **distributed mode** every worker exposes the same REST API (default `http://localhost:8083`) and the cluster coordinates config/offset/status through internal Kafka topics. **Creating a connector — POST /connectors.** The request body is JSON with exactly two top-level fields: ``` { "name": "my-source", "config": { "connector.class": "io.confluent.connect.jdbc.JdbcSourceConnector", "tasks.max": "1", "connection.url": "jdbc:postgresql://...", "topic.prefix": "db-" } } ``` Key points: - **`name`** identifies the connector cluster-wide and must be unique. POSTing a name that already exists returns **409 Conflict**. - **`config`** is a flat map of **string-to-string** values — even numbers like `tasks.max` are JSON strings. - **`connector.class`** is mandatory and selects the connector plugin (must be on the worker's plugin path). - A successful create returns **201 Created** with the connector's current `{name, config, tasks, type}`. **Alternative: PUT /connectors/{name}/config.** This endpoint is idempotent and upsert-style: the body is just the flat config map (no `name`/`config` wrapper). If the connector doesn't exist it is created (201); if it does it is reconfigured (200). This is the preferred form for declarative tooling (GitOps, scripts) because re-applying the same config is safe and doesn't 409. **Edge cases:** Connect validates the config before accepting it; a config that fails validation returns **400 Bad Request** with the per-field error messages (the same structure returned by PUT /connector-plugins/{class}/config/validate). The REST request is handled by whichever worker received it, which then forwards/coordinates with the cluster leader, so the create is durable across the whole Connect cluster, not just one worker.

  • What happens if you POST the same connector name twice?
    The second POST returns 409 Conflict because connector names must be unique. To change an existing connector use PUT /connectors/{name}/config instead, which updates in place.
  • What is the difference in body shape between POST /connectors and PUT /connectors/{name}/config?
    POST /connectors wraps the settings: {name, config:{...}}. PUT /connectors/{name}/config takes only the flat config map directly (no name or config wrapper), and the name comes from the URL path.

saying these in an interview costs you the question

  • Claiming the config can be a nested object — config is a flat string-to-string map.
  • Saying POST is idempotent — POST 409s on duplicate name; PUT /config is the idempotent path.
  • Forgetting connector.class is mandatory.
  • Thinking Connect runs on port 9092 (that's the broker; Connect's REST API is 8083).

context

resolving links…

open as a page

Explain the difference between GET /connectors/{name}/status, /config, and /tasks. What does the status endpoint tell you that /config does not?

level: middleimportance: must knowfreq 70%

basics

~20 s

/config returns the desired configuration; /tasks returns the per-task configs; /status returns runtime health: the connector's RUNNING/FAILED/PAUSED state plus each task's state, assigned worker, and any failure trace. Status is live; config is static intent.

open as a page

Walk through pausing, stopping, and resuming a connector via the REST API. How do PUT /pause, PUT /stop, and PUT /resume differ?

level: middleimportance: should knowfreq 50%

basics

~10 s

PUT /connectors/{name}/pause halts processing but keeps tasks assigned (state PAUSED). PUT /connectors/{name}/stop also halts but de-allocates the tasks (state STOPPED), freeing cluster resources. PUT /connectors/{name}/resume restarts a paused or stopped connector back to RUNNING.

open as a page

How do you keep secrets like passwords out of a connector config submitted to the REST API? Explain ConfigProvider externalization.

level: seniorimportance: should knowfreq 40%

basics

~20 s

Use a ConfigProvider: put a placeholder like ${file:/path:key} or ${vault:...} in the connector config instead of the literal secret. Connect resolves it at runtime from the provider, so the secret never lives in the REST payload or the config topic, and GET /config shows the placeholder.

open as a page

How does Kafka Connect validate a connector configuration before accepting it, and how does PUT /connector-plugins/{class}/config/validate help you build safe deployment tooling?

level: seniorimportance: should knowfreq 35%

basics

~20 s

Connect runs each connector's Validator over the proposed config. PUT /connector-plugins/{class}/config/validate returns a structured per-field result (definitions, current values, recommended values, and per-field errors) without creating anything, so tooling can pre-flight a config and surface errors before deploying.

open as a page

Describe the connector restart endpoint and the includeTasks and onlyFailed query parameters. How would you restart just the failed tasks of a connector?

level: seniorimportance: should knowfreq 45%

basics

~10 s

POST /connectors/{name}/restart restarts only the Connector instance by default. Add ?includeTasks=true to also restart tasks, and ?onlyFailed=true to limit the restart to FAILED instances. To restart just failed tasks: POST /connectors/{name}/restart?includeTasks=true&onlyFailed=true.

open as a page