When should you build an Airbyte connector with the low-code YAML manifest instead of the Python CDK?

## Two ways to write the same connector Airbyte's CDK offers two authoring styles that both produce a connector satisfying the same four-command protocol. The **low-code declarative** style is a `manifest.yaml` file interpreted at runtime by a declarative source: you describe *what* the API looks like and the framework does the calling, paging, retrying, extracting and checkpointing. The **Python CDK** style is code: you subclass stream classes and implement methods. The Connector Builder in the Airbyte UI is a front end for the first one — it edits a manifest and gives you a test-read panel that shows the request it sent, the raw response and the records extracted, which shortens the loop enormously compared with rebuilding a Docker image. ## What the manifest covers well A declarative stream is composed of a retriever holding three parts. The **requester** owns `url_base`, `path`, `http_method`, `request_parameters` and the authenticator. The **record selector** owns the extractor, normally a `DpathExtractor` with a `field_path` like `["data", "items"]`, which is how you say "the records are under this key". The **paginator** owns how to get the next page. On top of that sit an authenticator (bearer token, API key in a header or query parameter, OAuth2 with a refresh token), an incremental cursor (`DatetimeBasedCursor` with `cursor_field`, `datetime_format`, `start_datetime` and a `step` for windowed reads), and a partition router for streams that must be read per parent record. Interpolation strings like `{{ config['api_key'] }}`, `{{ stream_partition.id }}` and `{{ response.next_cursor }}` glue the pieces to runtime values, and `$parameters` plus YAML anchors let you factor the repetition out across twenty near-identical streams. That covers a startling share of SaaS APIs. Most REST APIs are: authenticate with a header, GET a collection, read records from one JSON key, follow a cursor or offset, filter by an updated-at parameter. For those, YAML removes an entire category of work — you are not writing, testing or upgrading Python. ## Where the manifest runs out The honest boundaries are: - **Not HTTP.** Databases, object stores, message queues and SDK-only vendors are Python territory; the declarative components assume an HTTP request/response. - **Response needs decoding, not extraction.** A path extractor navigates JSON. XML, CSV, NDJSON inside a gzip, or a payload where records must be reshaped or joined before emission wants code. - **Stateful or multi-step request flows.** Report APIs that make you POST a job, poll for completion, then download a file are awkward to express as a simple retriever, and custom request signing (an HMAC over the body, a rotating nonce) has no declarative component. - **Genuinely odd pagination or error semantics.** APIs that page by mutating a filter, or that signal end-of-data with a 404, will fight the standard strategies. A useful smell test: if you find yourself writing a Jinja expression with real logic in it, you have outgrown the manifest. ## You do not have to choose globally The manifest supports custom components: a component block with a `class_name` pointing at a Python class in your connector package, used for a custom requester, extractor, paginator strategy or authenticator. That means the common case is "declarative manifest with one custom extractor", not "rewrite everything in Python". Preserving the declarative skeleton keeps most of the maintenance benefit. ## The maintenance argument, which is the real one Interviewers usually want the second-order answer. A YAML connector has a smaller blast radius when the CDK upgrades: you are consuming versioned components rather than subclassing implementation. It is readable by someone who is not a Python developer — an analyst can see which endpoint feeds which stream. And it is uniform: a fleet of thirty declarative connectors looks like one thing with thirty configurations, whereas thirty hand-written Python connectors are thirty codebases with thirty sets of quirks and thirty flavours of retry logic. Against that, YAML debugging is worse when it goes wrong — a failing interpolation gives you a less direct error than a stack trace — and you are limited to the components that exist. ## How to answer the decision question Start declarative. Prototype in the Connector Builder against the real API, because the test-read tells you within minutes whether the response shape fits an extractor and whether pagination terminates. If it fits, ship YAML. If one part resists, add a custom component. Reach for a fully hand-written Python connector when the source is not an HTTP API at all, or when the extraction is a genuine program rather than a description.