In Airflow's TaskFlow API, how does returning a value from an @task function create an XCom?

## What the decorator actually does `@task` is sugar over an operator. Decorating a function and calling it inside a `@dag` (or a `with DAG(...)` block) constructs a task in the graph whose execution calls your function. When the function returns, Airflow pushes the return value as an XCom under the reserved key `return_value`, exactly as `do_xcom_push=True` does for a classic operator. Nothing new is stored, and nothing new is stored anywhere else: it is still a row in the `xcom` table of the metadata database, still serialized, still subject to the same size limits. ```python from airflow.decorators import dag, task import pendulum @dag(start_date=pendulum.datetime(2024, 1, 1, tz="UTC"), schedule="@daily", catchup=False) def pipeline(): @task def extract() -> list[int]: return [1, 2, 3] @task def load(rows: list[int]) -> None: print(sum(rows)) load(extract()) pipeline() ``` ## Calling the function does not run it The line `load(extract())` is the part worth understanding. At DAG-parse time, `extract()` does not execute the body and does not return `[1, 2, 3]`. It registers a task and hands back an `XComArg` — a placeholder object that means "the `return_value` XCom of the task named extract, in whatever run is executing." Passing that placeholder into `load(...)` has two effects. First, Airflow records the dependency, so the scheduler knows `extract` must succeed before `load` is queued — this is why TaskFlow DAGs usually have no explicit bitshift operators. Second, at runtime the placeholder is resolved by performing the pull, and your `load` function receives the concrete value as a normal Python argument. The implicit push and the implicit pull are the same mechanism as the classic API; the decorator just writes both halves for you and, crucially, cannot forget the dependency edge the way a hand-written `xcom_pull` can. ## Returning more than one thing By default a returned dict is a single XCom. If downstream tasks want the pieces separately, use `multiple_outputs`: ```python @task(multiple_outputs=True) def extract() -> dict[str, str]: return {"path": "s3://bucket/2024-05-01/", "partition": "2024-05-01"} result = extract() load(result["path"]) ``` With `multiple_outputs=True`, Airflow pushes one XCom per top-level dict key, and `result["path"]` is itself a reference to just that key — so the downstream task pulls only the piece it needs rather than the whole dict. Airflow can infer `multiple_outputs` from a dict return type annotation, but stating it explicitly is clearer to the next reader. Note that only top-level keys are split; nesting is not flattened. ## Mixing with classic operators The two styles interoperate through the same reference object. Any classic operator exposes `.output`, which is the `XComArg` for its `return_value`: ```python fetch = BashOperator(task_id="fetch", bash_command="./fetch.sh") load(fetch.output) # TaskFlow task consuming a classic operator ``` and a TaskFlow reference can be handed into a classic operator's templated field. This matters in real codebases, which almost never are purely one style. ## Dynamic task mapping A reference can also be expanded. `some_task.expand(arg=upstream())` creates one mapped task instance per element of the upstream list, and each instance writes its own XCom row distinguished by map index. Pulling from a mapped task in the aggregate gives you the list of per-index values. The relevant point for this topic is that mapping does not change where XComs live — it multiplies the rows, so a mapping over thousands of elements multiplies the metadata-database traffic too. ## The trap: syntax hides cost Because TaskFlow makes passing values look like ordinary Python function calls, it invites people to pass ordinary Python objects — a DataFrame, a big list of dicts, a parsed file. The syntax is friendly; the storage is not. Every argument passed between `@task` functions is serialized into Airflow's metadata database and read back out. The discipline is identical to the classic API: return small facts and URIs, write the actual data to object storage or a warehouse table, and set `do_xcom_push=False` (or simply return `None`) when a task's output is not needed. ## What to say in an interview "TaskFlow doesn't add a new data channel — it generates the same XCom push and pull, plus the dependency edge, from the function call. The value still goes through the metadata database, so the size discipline is unchanged." That sentence shows you understand the abstraction rather than just using it.