Why does a GCN normalize the neighbour sum by node degree instead of summing raw feature vectors?
answer
- degree changes the scale
- hub nodes versus ordinary nodes
- sum versus average of neighbours
- one over the square root of both degrees
- self-loop counts in the degree
basics
~20 sA raw neighbour sum scales with how many neighbours a node has, so hubs produce huge activations and ordinary nodes tiny ones. Dividing each edge's contribution by sqrt(d_i * d_j) keeps every node's aggregate on a comparable scale.
solid answer
~40 sOne graph convolution layer rebuilds each node's vector from its neighbourhood, so the aggregation step sets the scale. If you just sum, an account with 3,000,000 followers gets an aggregate three million times larger than an account with one: the layer encodes degree as raw magnitude, and the scale compounds as you stack layers. The standard fix is symmetric normalization. Take `A~ = A + I` (adjacency plus self-loops), let `D~` be its degree matrix, and propagate with `D~^-1/2 A~ D~^-1/2`, so the edge from `j` into `i` carries weight `1 / sqrt(d~_i * d~_j)`. That damps the receiver's degree *and* the sender's, so a celebrity's message is discounted everywhere it appears. Row normalization `D~^-1 A~` is the simpler alternative, a plain neighbourhood average, but it only normalizes the receiver.
code
python · 13 linesimport math
def coef(d_i, d_j):
# symmetric-normalization edge weight; degrees already include the self-loop
return 1.0 / math.sqrt(d_i * d_j)
ordinary = 11 # 10 followers + self-loop
celebrity = 3_000_001 # 3,000,000 followers + self-loop
print("ordinary <- ordinary :", round(coef(ordinary, ordinary), 6))
print("ordinary <- celebrity:", round(coef(ordinary, celebrity), 9))
print("celebrity <- ordinary:", round(coef(celebrity, ordinary), 9))
print("row-normalized weight:", round(1.0 / ordinary, 6))go deeper
Be ready to state what one layer does: combine the neighbours' vectors, multiply by a weight matrix shared across all nodes, apply a nonlinearity — and say why the combining step divides by degree instead of just summing.
Explain the propagation matrix D^-1/2 (A + I) D^-1/2 term by term: why the sender's degree appears as well as the receiver's, what the added identity contributes, and how row normalization differs.
Show you notice the cost. Normalization erases degree, which is often the very signal you want in fraud or influence problems, so you decide deliberately whether to feed a degree feature back in.
Own the framing that the aggregation coefficient is a modelling assumption about the domain — how much a link from a broadcaster should count — and defend choosing it per graph rather than inheriting a default.
## The layer, in one line A graph convolution layer rebuilds every node's vector out of its own vector and its neighbours'. For node `i` with feature vector `h_i` and neighbour set `N(i)`: `h_i' = sigma( W^T * sum over j in N(i) U {i} of c_ij * h_j )` `W` is a weight matrix shared by every node in the graph, `sigma` is a nonlinearity, and `c_ij` is the coefficient the aggregation step gives the edge `j -> i`. This question is entirely about `c_ij`. ## What breaks when c_ij = 1 With `c_ij = 1` the aggregate is a plain sum, and its magnitude grows linearly with degree. In a follower graph, an account with 3,000,000 followers gets a vector roughly three million times longer than an account with one follower, before `W` is even applied. Three consequences, in order of how much they hurt: 1. **Degree leaks in as magnitude.** Two nodes with identical neighbourhoods in *composition* but different neighbourhood *size* get wildly different embeddings. Whatever the classifier learns downstream, it is partly reading degree off the vector norm — and it reads it in an uncontrolled, unnormalized way. 2. **Scale compounds with depth.** Stack two layers and the hub's aggregate has been multiplied by a degree-sized factor twice. Activations for hubs blow up while leaf nodes stay near zero, so a single learning rate cannot suit both, and gradients are just as skewed. 3. **Optimization gets miserable.** Batches mix nodes whose activations differ by orders of magnitude; a rectifier passes the blow-up straight through, and saturating nonlinearities pin hubs at their ceiling. ## The two normalizations **Row normalization**, `D~^-1 A~`, sets `c_ij = 1 / d~_i` for every neighbour. The coefficients sum to exactly one, so the aggregate is a genuine average of the neighbourhood: "what does a typical neighbour look like". It normalizes the *receiver* only. **Symmetric normalization**, `D~^-1/2 A~ D~^-1/2`, sets `c_ij = 1 / sqrt(d~_i * d~_j)`. Here the *sender's* degree matters too. A celebrity node reaches every one of its followers with a coefficient shrunk by `1/sqrt(3,000,001)`, so a message from a node that broadcasts to millions counts for very little in any single recipient's update — the intuition being that a link from someone who links to everyone carries less information than a link from someone selective. The weights no longer sum to one, so this is not literally an average; what it buys is a propagation matrix whose largest eigenvalue is at most 1, so repeated application does not amplify the signal. Both are defensible. Symmetric is the common default in the classic two-layer setup; row normalization is easier to explain and is the natural choice if you genuinely want "the mean neighbour". ## Why the self-loop is inside the normalization `A~ = A + I` puts the node itself into its own neighbourhood, so `h_i` survives the aggregation instead of being discarded. It has a second, mundane benefit: an isolated node has degree zero in `A`, which makes `D^-1` and `D^-1/2` undefined. With the self-loop its degree is one, the row is well defined, and the node simply passes its own features through. Note that `d~_i = d_i + 1` — the degrees used in the coefficients include the self-loop. ## A worked setting On a citation network — classify each paper's topic from a bag-of-words feature vector, with roughly 20 labelled papers per class and a two-layer network over the whole graph — normalization is what makes the labelled signal spread usefully. A heavily cited survey paper has hundreds of edges; unnormalized, it would dominate the aggregate of every paper that cites it and drag their representations toward one blurry point. Normalized, it contributes in proportion to how selective the link is. ## The cost you should name Normalization deliberately erases degree from the aggregate's scale. Sometimes degree is exactly the signal — follower count for influence, transaction count for fraud, citation count for impact. The right move is not to skip normalization; it is to put degree back in as an explicit node feature (typically `log(1 + degree)`, so the value range is sane) and let the model learn what to do with it. ## Misconceptions worth avoiding - "The nonlinearity will handle the scale." A rectifier is unbounded and passes the blow-up through unchanged. - "Sum and average differ by a constant, and `W` can absorb it." The constant is per-node — it is the degree — so no single weight matrix can absorb it. - "Symmetric normalization means dividing by the node's own degree." That is row normalization; the symmetric form takes the geometric mean of both endpoints' degrees. - "Normalization is only about avoiding numerical overflow." It is mainly about not letting degree masquerade as feature magnitude.
- What does row normalization D^-1 (A + I) do differently from the symmetric version?Row normalization gives every neighbour weight `1/d~_i`, so the coefficients sum to one and the aggregate is a true neighbourhood mean — but it only accounts for the receiving node's degree. The symmetric form uses `1/sqrt(d~_i * d~_j)`, which additionally discounts messages from high-degree senders, so a node that links to everyone influences each recipient less.
- If node degree is genuinely predictive in your domain, how do you keep that signal after normalizing?Add it back as an explicit feature. Append `log(1 + degree)` — and any other structural statistic you trust, such as a clustering coefficient — to each node's input vector. The propagation stays scale-stable, and the model can learn a deliberate, bounded use of degree instead of reading it off an activation's magnitude.
- What breaks if a node has no neighbours and you skip the self-loop?Its row of the adjacency matrix is all zeros, so its aggregate is the zero vector and its own features never reach the next layer. Worse, its degree is zero, so `D^-1` and `D^-1/2` are undefined. Adding `I` gives it degree one and it simply propagates itself.
Reading a recommendation letter: one from a referee who writes ten a year says more than one from a referee who signs a million. Symmetric normalization discounts by how much both the writer and the reader are involved.
saying these in an interview costs you the question
- Says the nonlinearity will absorb the scale difference
- Claims summing and averaging give the same embedding up to a constant
- Divides only by the receiving node's degree and calls it symmetric
- Forgets the self-loop, so a node's own features disappear
- Treats normalization as purely a numerical-overflow guard
- Ignores that normalization throws away a possibly predictive degree signal