Why does GraphSAGE concatenate a node's own vector with the neighbour aggregate?
answer
- own features versus neighbourhood features
- the self-loop mixes them together
- separate columns of the weight matrix
- self weight shrinks as degree grows
- keep the two channels separable
basics
~20 sConcatenation keeps a node's own features in their own slots, so the layer learns separate weights for what the node looks like and what its neighbours look like. Averaging blends the two signals into a single inseparable one.
solid answer
~40 sA GraphSAGE layer computes `h_i <- sigma( W * concat(h_i_prev, AGG(neighbours)) )`, so the self half and the neighbour half land in different columns of `W` and get independent learned transforms — the model can even give them opposite signs. A graph convolution instead folds the node in as one more neighbour through the self-loop, so self and neighbour features share one weight matrix and the self contribution carries weight around `1/(d_i + 1)`: the more neighbours a node has, the more its own profile is diluted. For a suspected spam account that distinction matters, because 'this profile looks like spam' and 'this account's neighbours look like spam' are separately actionable signals. The cost is that the layer's input width doubles.
go deeper
Know that each layer combines a node's own feature vector with a summary of its neighbours, and that there are two ways to do it: mix them into one aggregate, or keep them side by side.
Be able to write the concatenated update and split the weight matrix into a self block and a neighbour block, then explain what that split lets the layer represent that a blended aggregate cannot.
Argue it from the data: on a fraud or spam graph, say which decisions depend on the node's own profile versus its neighbourhood, and note that the blended form's self weight decays with degree.
Own the tradeoff between expressiveness and parameter count across the whole model family, and set the default your team starts from given the graph's homophily and how much labelled data exists.
## Two ways to keep the node itself in its own update Every message-passing layer has to decide what to do with the node's own vector, and there are exactly two common answers. **Blend it in.** Add a self-loop so the node appears in its own neighbourhood, then aggregate everything together: `h_i' = sigma( W^T * sum over j in N(i) U {i} of c_ij * h_j )` **Keep it separate.** Aggregate only the neighbours, then concatenate: `h_i' = sigma( W * concat( h_i , AGG({h_j : j in N(i)}) ) )` The second is what GraphSAGE does, and the difference is not cosmetic. ## What concatenation actually buys **Separate learned maps.** Write `W = [W_self | W_neigh]`. Then `W * concat(h_i, a_i) = W_self * h_i + W_neigh * a_i`. The self vector and the neighbourhood aggregate each get their own linear transform. The model can amplify one and suppress the other, project them onto different subspaces, or learn opposite signs — "flag me when I look clean but my neighbours do not" is a representable function. Under the blended rule, both go through the *same* `W`, so the layer sees only one quantity: a weighted sum of self and neighbour features. Any information about which part of the mix came from where is gone before `W` is applied, and no choice of `W` can recover it. **A degree-independent self weight.** In the blended rule the node's own coefficient is `c_ii`, which is `1/(d_i + 1)` under row normalization or `1/(d_i + 1)` under the symmetric form as well (both endpoints are `i`). Either way it shrinks as the node's degree grows. A node with five neighbours keeps about a sixth of its own signal; a node with five hundred keeps about a five-hundredth. Concatenation gives the self vector a fixed slot whose weight is learned, not dictated by degree. ## The concrete case Consider scoring accounts for spam. Two distinct signals exist: the account's own profile (age, posting rate, bio text) and the profile of the accounts it interacts with. A spam ring where every member looks clean individually is caught by the neighbourhood half. A lone spammer with hijacked legitimate connections is caught by the self half. If the layer averages the two together, a clean-looking spammer with clean-looking neighbours and a clean account with one bad neighbour can produce similar aggregates, and "I look like this" versus "I look like my neighbours" stop being separable. The concatenated form keeps both channels open into the next layer. ## What it costs - **Parameters and width.** If both halves are `F`-dimensional, `W` is `2F x F'` instead of `F x F'` — double the parameters in that layer, with the usual overfitting risk when the labelled set is small. - **Memory.** The concatenated activations are twice as wide before the projection, which matters when neighbourhood aggregates are materialized for many nodes at once. - **Less smoothing.** Blending is a smoothing operator, which is exactly what you want on a strongly homophilous graph where a node really is well described by its neighbourhood. In that regime the blended variant is a reasonable, cheaper choice, and GraphSAGE offers such a variant that averages the self vector in rather than concatenating it. ## A detail that often accompanies it GraphSAGE normalizes each layer's output embedding to unit L2 norm. That keeps representations on a common scale regardless of how many neighbours contributed, makes dot-product similarity between node embeddings meaningful, and stops magnitude differences from accumulating across layers. It is a separate mechanism from concatenation, but it addresses the same underlying nuisance: neighbourhood size should not silently set the size of a vector. ## Misconceptions worth avoiding - "Concatenation and averaging differ only by a scale factor." They differ in what is *representable*: averaging destroys the self/neighbour split before any weights are applied. - "A graph convolution cannot see the node's own features." It can — that is what the self-loop is for. The point is that it sees them mixed with the neighbours' and weighted by degree. - "Concatenation is free." It doubles the layer's input width and the parameters of that projection. - "Concatenation is always better." On a homophilous graph with few labels, the extra parameters can hurt more than the extra expressiveness helps.
- What does concatenation cost compared with averaging the self vector in?The layer's input width doubles, so the projection matrix goes from `F x F'` to `2F x F'` — twice the parameters and twice the activation width to hold. On a small labelled set that extra capacity can overfit, which is why a blended variant remains a sensible baseline on strongly homophilous graphs.
- GraphSAGE normalizes each layer's output embedding to unit length — what does that buy?It puts every node's representation on a common scale regardless of neighbourhood size, so magnitude differences do not accumulate across layers and dot-product or cosine similarity between node embeddings is meaningful. It is a scale fix, not a substitute for concatenation, which is about keeping self and neighbour signals separable.
- When would blending the self vector in be the better choice?On a strongly homophilous graph, where a node really is well described by its neighbourhood, the blend acts as useful smoothing and halves the projection's parameters. With only a handful of labels per class, that regularizing effect can beat the extra expressiveness of a concatenated layer.
A candidate's own resume and their references, stapled together and read separately, versus blended into one averaged summary paragraph. The staple lets you weigh each independently, and notice when they disagree.
saying these in an interview costs you the question
- Says concatenation and averaging differ only by a constant factor
- Thinks a self-loop gives the node's own features a degree-independent weight
- Claims concatenation adds no parameters to the layer
- Believes a graph convolution cannot use a node's own features at all
- Cannot name a case where blending the self vector is preferable