When must a custom Haystack component implement to_dict and from_dict?

## What serialization is for Haystack lets you turn a pipeline into a dict or a YAML file and load it back somewhere else. That is what makes a pipeline a reviewable artifact rather than a pile of glue code: you can diff it, promote it between environments, or let a non-Python tool render it. Component serialization is the unit that makes it work — a pipeline's serialized form is essentially its components' serialized forms plus the connections between them. ## The serialized shape Every component becomes: ``` { "type": "my_package.filters.TagFilter", "init_parameters": {"allowed": ["a", "b"]} } ``` `type` is the fully qualified import path of the class. `init_parameters` is what will be passed to `__init__` on the way back. Note what is *not* there: no run-time state, no loaded model, no outputs. Serialization captures how the component was configured, and reconstruction rebuilds it from scratch. ## The default path and its two requirements If you implement nothing, Haystack uses `default_to_dict`/`default_from_dict`. For that to work: 1. **1:1 naming.** Each `__init__` parameter must be stored on an attribute of the same name. If your constructor takes `allowed` and you store `self._allowed`, discovery breaks. 2. **JSON-expressible values.** Strings, numbers, booleans, lists, dicts and `None` round-trip. Everything else does not. A component whose configuration is a model name and a `top_k` needs no serialization code at all, and most custom components are in that category. This is worth saying explicitly in an interview, because the reflexive answer — "always implement them" — is wrong and adds noise to every component you write. ## When you must intervene The common triggers: - **A `set`.** JSON has no set. Convert to a sorted list on the way out, back to a set on the way in. Sorting is not cosmetic: it makes the serialized form deterministic, so a re-export does not produce a spurious diff. - **A callable.** A streaming callback or a filter function cannot be embedded. Haystack ships `serialize_callable`/`deserialize_callable`, which record the callable's import path and resolve it again. A lambda or a closure has no import path and cannot be serialized at all — that is a real design constraint on your component's API. - **A `Secret`.** Credentials serialize as a reference; on the way back, `deserialize_secrets_inplace(data["init_parameters"], keys=["api_key"])` rebuilds the object before construction. - **Nested components or tools.** Anything that itself has a serialized form has to be serialized recursively rather than embedded raw. - **Custom objects generally** — a config dataclass, a device descriptor, an enum — need a chosen wire form and a conversion each way. ## The pattern ``` def to_dict(self): return default_to_dict(self, allowed=sorted(self.allowed)) @classmethod def from_dict(cls, data): data["init_parameters"]["allowed"] = set(data["init_parameters"]["allowed"]) return default_from_dict(cls, data) ``` Note the asymmetry that trips people: `default_to_dict(self, **params)` takes the parameters as keyword arguments, while `from_dict` mutates `data["init_parameters"]` in place and then hands the *whole* `data` dict to `default_from_dict`. Reaching for `data["allowed"]` instead of `data["init_parameters"]["allowed"]` is the single most common mistake in this code. ## The import-path trap Because `type` is an import path, the loading process must be able to import that exact path. A component class defined in a Jupyter notebook, in a `__main__` script, or inside a function body will serialize happily and then fail to load in a worker, a test process, or another service. The fix is not clever — move the class into an installable module — but the failure is confusing enough that it is worth naming before it costs an afternoon. A related discipline: renaming or moving a component class breaks every stored pipeline file that references it. Once definitions are checked in, class paths are part of your contract with those files. ## Configuration, not state `to_dict` must describe how to rebuild the component, which means it reports the values it was constructed with. If a component loads a model in `warm_up()` and stores it on an attribute, that attribute must not appear in the serialized form — and if `warm_up()` mutates one of the attributes `to_dict` reads, a warmed instance will serialize differently from a cold one and your round-trip stops being faithful. Keeping the serialized surface identical to the constructed surface is the invariant to hold, and a good test for a custom component is exactly that: construct, serialize, deserialize, serialize again, and assert the two dicts are equal.