Inductive Training at Scale

## The question behind the words "Transductive" and "inductive" answer one thing: **which nodes was the model allowed to see while it was being fitted, and which nodes can it therefore score afterwards?** Everything else follows from that. ### Transductive training In the transductive setting there is one fixed graph. Training runs over the whole adjacency at once, labels exist for some nodes, and the goal is to fill in labels for the *other nodes of that same graph*. Crucially, the unlabelled nodes were present the entire time — their edges participated in every message-passing step, and their features (if any) flowed into their neighbours' representations. A model becomes hard-transductive when it carries **free parameters indexed by node identity**: a table with one trainable vector per node, learned directly by gradient descent. Nothing in that table generalises. A node that did not exist when the table was allocated has no row, and there is no forward pass that can produce one. The only remedy is to refit. A model can also be *softly* transductive through its training procedure rather than its parameters. Full-graph training that normalises by global quantities, or an evaluation that leaves test nodes and their edges inside the training graph, both assume the graph is closed. ### Inductive training In the inductive setting the model's parameters are **shared functions**, not per-node values: weight matrices that take a node's own feature vector plus aggregated feature vectors from its neighbours and return a representation. Because the same weights are applied at every node, they apply just as well to a node the model has never seen. Scoring a new node needs only two things at inference time: its features, and access to its current neighbours. That second requirement is the one people forget. An inductive model is not context-free — it still has to read the neighbourhood. A brand-new node with no edges yet falls back to whatever the architecture does with an empty neighbourhood, usually just its own transformed features, which is a much weaker signal. ### Three degrees of "unseen" It helps to grade the difficulty: 1. **Unlabelled but present.** The node was in the training graph; only its label was withheld. This is semi-supervised *transductive* node classification. It is the easiest case and the one most benchmark numbers report. 2. **New node, same graph.** The node was created after the training run — a new account, a new product, a new device. The model must produce a representation from features and current neighbours alone. 3. **New graph entirely.** The strongest form. A protein–protein interaction benchmark has exactly this shape: fit on around twenty tissue graphs, tune on a couple more, then run on a tissue graph the model has never touched, whose node identities mean nothing across graphs. Only shared, feature-driven weights survive that transfer. ### Why this bites in production Consider a feed-ranking graph rebuilt daily. Accounts created since last night's training run must be scored tonight; there is no retraining window before the traffic arrives. If the ranking model is transductive, those accounts are unscorable and fall into a cold-start default path, and the cost of being wrong compounds because new accounts are exactly the population you most want to rank correctly. If it is inductive, tonight's scoring job builds each new account's representation from its profile features and the neighbours it has accumulated today, using the same weights fitted last week. Retraining then becomes a periodic quality refresh rather than a hard dependency of serving. ### The evaluation trap The most common way to fool yourself is to *claim* inductive evaluation while measuring the transductive case. If your held-out nodes were present in the graph during training — their edges intact, their features feeding neighbours — then information about them leaked into the fitted weights and into every neighbour representation. A genuine inductive evaluation **removes the test nodes and all their incident edges from the training graph**, fits, then reinserts them only at inference. The gap between the two protocols is often large, and discovering it after launch is expensive. ### How to tell which one you have Ask one question of the architecture: *if a node ID appeared that was never in the training set, what would the forward pass do?* If the answer is "index a table and fail", it is transductive. If the answer is "transform its features and aggregate its neighbours' features", it is inductive. Mixed designs exist — feature-driven weights plus a learned per-node correction — and they inherit the transductive limitation for any node whose correction does not exist.