In a graph-level GNN, what does a sum or mean readout over the node vectors do?

## The shape problem A message-passing body produces a matrix of node vectors: `N x d`, where `N` is the number of nodes in *this* graph. A graph-level task — is this molecule toxic, does this route plan hold together, is this program correct — needs one prediction per graph, so the head needs one vector of fixed width. Two facts make this non-trivial: 1. `N` differs from graph to graph. A dataset of molecules ranges from a handful of atoms to hundreds. 2. Node indices are arbitrary. The same molecule written with its atoms listed in a different order is the same molecule. The **readout** (also called pooling, or aggregation over the graph) is the function that maps the `N x d` node set to a single `d`-dimensional graph vector, and it has to respect both facts. ## Permutation invariance is the hard requirement A readout `R` must satisfy `R(h_1, ..., h_N) = R(h_pi(1), ..., h_pi(N))` for every reordering `pi`. If it does not, the model gives different answers for the same graph depending on how the file happened to list the nodes, and it can never learn its way out of that — it would have to memorise every ordering. This immediately disqualifies the two things people reach for first: - **Concatenating the node vectors** and padding to a maximum size. The output depends entirely on order, and it wastes most of its width on small graphs. - **Sorting nodes by degree** (or by any feature) to fix an ordering. It is fragile — ties are common and arbitrary, and a tiny perturbation of the graph can reshuffle the whole vector, so the function is discontinuous in the input. Sum, mean, min and element-wise max are all invariant by construction, which is why they are the standard readouts. ## Sum versus mean versus max `sum`: `g = sum over v of h_v`. - Retains **how many** nodes there are, and how many of each kind. Two graphs made of two copies versus four copies of the same motif give different sums. - It is the most expressive of the three over multisets — the theoretical result behind Graph Isomorphism Networks is that a sum aggregator can in principle be injective over multisets of features, while mean and max cannot. - Its magnitude grows with graph size. A model trained mostly on 20-node graphs and shown a 300-node graph produces activations far outside the range it saw in training, and the downstream layers extrapolate badly. `mean`: `g = (1/N) sum over v of h_v`. - Size-normalised, so magnitudes stay comparable across a dataset with a very wide size range, which usually makes optimisation easier. - Discards the count entirely. Two copies and four copies of the same motif average to the same vector. - That is exactly the wrong choice when the size itself is predictive. On a molecular toxicity screen where larger molecules are more often flagged, mean throws away a genuinely informative feature; sum keeps it, at the price of the extrapolation problem above. `max` (element-wise): `g[i] = max over v of h_v[i]`. - Detects **presence**: does any node exhibit this feature strongly? Excellent when one salient substructure decides the label. - Loses both multiplicity and size, and passes gradient to only one node per coordinate, which can slow learning. ## Practical resolutions - **Concatenate several readouts.** `[sum ; mean ; max]` gives the head all three views at three times the width and costs nothing conceptually. This is a very common default when you do not want to choose. - **Mean plus an explicit size feature.** Keep the stable magnitude of the mean and hand the node count (or its logarithm) to the head as an extra input, so size is available as a feature rather than as a scale factor. - **Learned weighted pooling.** Score each node with a small shared function, normalise the scores across the graph, and take the weighted sum. Because the same function is applied to every node and the combination is a sum, the result stays permutation invariant while letting the model focus on the nodes that matter. It adds parameters and needs more data to pay off. - **Hierarchical pooling.** Coarsen the graph in stages, pooling clusters of nodes before the final readout, when the task depends on structure at several scales. ## The diagnostic to remember If a graph-level model does well on typical graphs and degrades sharply on the largest ones, look at the readout first. A sum readout makes the graph vector's norm scale with `N`, so the largest graphs sit in a region of activation space the head never trained on. Switching to mean plus a size feature, or normalising the pooled vector, usually recovers it — and tells you whether the loss was information or scale. ## What to say when asked Define the readout as the permutation-invariant reduction from a variable-sized node set to one graph vector, name sum, mean and max, say why concatenation and sorting are disqualified, and then make the sum-versus-mean tradeoff concrete with a case where graph size is itself predictive.